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CHAPTER  I 


INTRODUCTION 

The  Department  of  Defense  systems  acquisition  pro¬ 
cess  consists  of  a  sequence  of  activities  originating  with  an 
agency's  identification  of  mission  need  and  extending  through 
the  introduction  of  a  system  into  operational  use  or  the 
successful  completion  of  program  objectives  (24:3).  Incorpor¬ 
ated  within  this  framework  is  the  contracting  process,  which 
is  a  subordinate  function  occurring  within  the  overall  acquisi¬ 
tion  process.  An  essential  activity  during  the  contracting 
process  is  communication  with  industry.  Early  exchange  of  in¬ 
formation  can  enhance  the  actual  formulation  of  acquisition 
needs.  One  important  method  of  communicating  with  industry  is 
the  Draft  Request  for  Proposal  which  solicits  feedback  on 
planned  acquisitions.  Such  feedback  has  possessed  the  poten¬ 
tial  for  significant  cost  savings  and  program  improvements  by 
eliminating  unnecessary  requirements  and  overly  complex 
elements  (21:1). 

Statement  of  the  Problem 

Air  Force  Systems  Command  (AFSC)  initiated  the  Draft 
Request  for  Proposal  (DRFP)  concept  in  1977  upon  completion  of 
a  six-month  study  which  tested  its  use.  AFSC  had  long  appre¬ 
ciated  the  need  for  acquiring  industry's  assistance  in  solving 
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and  defining  acquisition  needs.  The  Command  had  extended 
solicitation  of  industry  feedback,  which  applied  to  major 
systems  acquisition,  to  other  programs  not  classified  as  major 
systems.  AFSC  had  primarily  solicited  the  technical  expertise 
and  knowledge  of  industry  by  requesting  voluntary  review  of 
solicitations  (7:1).  With  the  initiation  of  DRFPs,  AFSC  began 
a  structured  solicitation  of  feedback  on  planned  acquisitions. 

Many  AFSC  acquisitions  have  demonstrated  successful 
application  of  the  DRFP  concept;  however,  several  problems  in 
its  use  have  been  identified.  The  former  commander  of  AFSC, 
General  Alton  Slay,  has  expressed  concern  over  failure  of  con¬ 
tracting  officers  to  coordinate  DRFPs  with  the  contract  admini 
stration  offices  of  prospective  contractors  when  the  DRFP  is 
released  to  industry  (26:1).  Such  failure  prevents  the  oppor¬ 
tunity  to  provide  early  assistance  in  eliminating  proposal 
problems  and  can  cause  delays  in  the  acquisition  process. 
Additionally,  during  an  interview  in  the  Contract  Review  Divi¬ 
sion,  Directorate  of  Research  and  Development  Contracting, 

Mr.  Carl  Brigner  observed  that  there  are  inconsistencies  in 
present  DRFP  practices  which  deteriorate  the  effectiveness  of 
the  concept,  such  as  different  acknowledgments  by  contracting 
officers  of  industry  responses  (9).  Finally,  Major  Robert 
Golden,  Deputy  Director  of  the  Air  Force  Business  Research 
Management  Center,  has  expressed  the  need  for  a  study  of  the 
DRFP  concept  to  provide  a  thorough  assessment  of  its  effective 
ness  (13).  Such  a  study  is  the  focus  of  this  research  effort. 
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Background 

The  systems  acquisition  process  consists  of  four 
milestone  decisions  and  four  phases  of  activity  directed  to¬ 
ward  the  acquisition  of  a  system.  Milestone  0  is  distinguished 
by  Secretary  of  Defense  approval  of  the  Mission  Element  Needs 
Statement  (MENS) .  Department  of  Defense  components  prepare 
the  MENS  to  document  major  deficiencies  in  ability  to  meet 
mission  requirements.  Approval  of  the  MENS  authorizes  initi- 
ation  of  the  first  phase- -Concept  Exploration;  this  phase 
focuses  upon  alternative  systems  exploration.  During  Concept 
Exploration,  a  program  manager  is  designated;  this  person  is 
responsible  for  the  management  and  development  of  strategy 
for  the  potential  acquisition.  The  program  manager's  role  and 
acquisition  strategy  encompass  the  entire  acquisition  process. 
Milestone  I  decision  is  marked  by  selection  of  alternative 
systems  and  authorization  to  proceed  into  the  second  phase, 
which  is  Demonstration  and  Validation.  Activities  during  this 
phase  are  intended  to  verify  that  the  chosen  alternatives  are 
sound  and  can  perform  in  an  operational  environment.  These 
activities  establish  the  basis  for  selection  of  an  alternative 


system(s)  to  be  continued  in  full-scale  development.  Milestone 
II  decision  is  the  selection  of  the  alternative(s)  and  authori¬ 
zation  to  proceed  to  the  third  phase,  Full-Scale  Development. 
This  phase  provides  limited  prototypes  for  evaluation  in  an 
operational  environment.  Milestone  III  decision  authorizes 
progression  into  the  final  phase- -Production  and  Deployment. 

The  final  three  milestone  decisions  are  documented  by  the 


Decision  Coordinating  Paper  (DCP) .  This  document  is  reviewed 
by  the  Defense  Systems  Acquisition  Review  Council,  which  makes 
a  recommendation  to  the  Secretary  of  Defense  for  each  milestone 
decision  (18:10-16;  24:4-5). 

The  systems  acquisition  process  presents  the  overall 
framework  of  the  whole  spectrum  of  acquisition  activities  from 
recognition  of  a  need  to  fulfillment  of  that  need.  Closely 
related  to  this  framework  is  the  contracting  process.  The 
first  phase  of  this  process  is  the  requirements  generation 
phase,  which  is  initiated  by  program  approval.  Formal  identi¬ 
fication  of  requirements  is  documented  in  the  purchase  request. 
The  acquisition  planning  phase  comprises  activities  conducted 
in  response  to  the  purchase  request.  Issuance  of  solicitation 
to  prospective  offerors  marks  the  transition  from  acquisition 
planning  to  offer  evaluation.  Once  proposals  are  evaluated, 
the  decision  is  made  on  contract  award.  The  award  initiates 
the  final  phase  of  the  process- -contract  management.  Once 
the  acquisition  is  complete,  the  contracting  process  is  ter¬ 
minated  with  retirement  of  the  contract  file  (15).  The  acti¬ 
vities  of  the  contracting  process  occur  simultaneously  as  an 
integral  part  of  the  system  acquisition  process.  Many  of  the 
activities  of  both  processes  may  overlap,  and  several  procure¬ 
ment  processes  may  occur  throughout  the  acquisition  process 
(19:38) . 

The  focus  of  this  research  effort  is  the  acquisition 
planning  phase  of  the  contracting  process.  One  of  the  import¬ 
ant  decision  activities  of  the  phase  is  the  method  of 
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acquisition.  The  basic  choices  are  formal  advertising  and 
negotiation.  The  Armed  Services  Procurement  Act  of  1947  esta¬ 
blishes  formal  advertising  as  the  preferred  method  of  acqui¬ 
sition  in  the  DoD  arena.  Under  formal  advertising,  contract 
award  is  made  to  the  responsible  bidder  who  offers  the  lowest 
price,  provided  that  the  bid  is  considered  reasonable  and  most 
advantageous  to  the  Government  (10:554).  In  contrast  to  for¬ 
mal  advertising,  negotiation  provides  more  freedom  and  flexi¬ 
bility  to  the  contracting  officer  to  award  the  contract  to  the 
offeror  whose  proposal  appears  most  qualified.  Even  though 
Congress  established  formal  advertising  as  the  rule,  approxi¬ 
mately  90  percent  of  defense  acquisitions  in  recent  years  have 
been  accomplished  by  negotiation  (10:554).  The  need  for  nego¬ 
tiation  is  frequently  found  in  research  and  development  of  new 
weapon  systems  or  production  of  highly  complex  systems  where 
thorough  communication  of  contract  proposals  is  essential  for 
clear  understanding  of  requirements  (8:92).  Thus,  Congress 
has  recognized  that  in  some  instances  it  is  not  practical  to 
accomplish  government  contracting  through  formal  advertising. 
The  rationale  for  choosing  negotiation  is  contained  in  the 
Defense  Acquisition  Regulation  by  the  following  17  exceptions 
to  formal  advertising: 

1.  A  national  emergency; 

2.  Public  exigency; 

3.  Purchases  not  more  than  $10,000; 

4.  Personal  or  professional  services; 

5.  Services  of  educational  institutions; 

6.  Purchases  outside  the  United  States; 

7.  Medicines  or  medical  supplies; 

8.  Supplies  purchased  for  authorized  resale; 

9.  Perishable  or  nonperishable  subsistence  supplies; 
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10.  Supplies  or  services  for  which  it  is  impractical  to 
secure  competition  by  formal  advertising; 

11.  Experimental,  developmental,  or  research  work; 

12.  Classified  purchases; 

13.  Technical  equipment  requiring  standardization  and 
interchangeability  of  parts; 

14.  Technical  or  specialized  supplies  requiring  sub¬ 
stantial  initial  investment  or  extended  period  of 
preparation  for  manufacture; 

15.  Negotiation  after  advertising; 

16.  Purchases  in  the  interest  of  national  defense  or 
industrial  mobilization; 

17.  Procurement  otherwise  authorized  by  law,  e.g. 
architectural  or  engineering  services  for  preparing 
specifications  for  public  works,  utilities,  naval 
vessels,  or  aircraft  construction  [23:3.5-3.17]. 

Under  negotiation,  the  acquisition  planning  phase 
includes  the  preparation  and  issuance  of  a  Request  for  Propo¬ 
sal  (RFP) .  The  RFP  provides  performance  and  design  require¬ 
ments  of  the  system  and  lists  specific  information  that  the 
contractor  must  provide  regarding  schedules,  costs,  logistics, 
quality  control,  testing,  management,  past  experience,  and 
personnel  (12:264).  The  RFP  package  consists  of  the  following 
four  parts: 

Part  I  -  the  Schedule 

Section  A  -  Contract  Form 
Section  B  -  Supplies/Services  and  Prices 
Section  C  -  Description/Specifications 
Section  D  -  Packaging  and  Marking 
Section  E  -  Inspection  and  Acceptance 
Section  F  -  Deliveries  or  Performance 
Section  G  -  Contract  Administration  Data 
Section  H  -  Special  Provisions 

Part  II  -  General  Provisions 

Section  I  -  General  Provisions 

Part  III  -  List  of  Documents,  Exhibits,  and  Other 
Attachments 

Section  J  -  List  of  Documents,  Exhibits,  and 
Other  Attachments 

Part  IV  -  General  Instructions 
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Section  K  -  Representation,  Certifications,  and 
other  Statements  of  Offeror 

Section  L  -  Instructions  and  Conditions,  and 
Notices  to  Offerors 

Section  M  -  Evaluation  Factors  for  Award  [23:3.59]. 

The  RFP  is  considered  one  of  the  most  important  docu¬ 
ments  in  the  acquisition  process;  however,  as  demonstrated  by 
the  list  of  parts  of  the  RFP  package,  the  potential  for  numer¬ 
ous  complexities  and  voluminous  size  is  ever  present.  In  1969, 
the  Air  Force  Requests  for  Proposal  Study  evaluated  the  use  of 
RFPs  and  recommended  limiting  the  number  of  pages  and  carefully 
screening  the  requirements  placed  on  prospective  contractors 
for  data,  certificates,  and  management  reports  (14:1). 

Faced  with  the  potential  complexities  and  desire  to 
make  the  RFP  more  effective,  AFSC  initiated  the  DRFP  concept. 
Its  purpose  is  solicitation  of  industry  feedback  that  may 
provide  significant  cost  savings  and  program  improvements. 

Areas  which  may  have  potential  for  cost  savings  include: 

(i)  Changes  to  requirements  specifying  manufacturing 
processes,  facilities,  tools,  and  test  capabilities, 

(ii)  Schedule  or  delivery  changes  that  will  result 
in  cost  reductions, 

(iii)  Substitution  of  RFP-stated  materials,  and  overly 
restrictive  engineering  and  quality  specifications  and 
requirements , 

(iv)  The  use  of  common  Government -furnished  equipment 
or  standard  commercial  components  instead  of  system 
peculiar  items, 

(v)  Reduction,  substitution,  or  deletion  of  military 
specifications  and  standards, 

(vi)  Excessive  reporting  requirements, 

(vii)  Modified  contractual  arrangements  designed  to 
financially  incentivize  contractors  and  subcontractors 
to  pursue  cost-reduction  efforts  during  contract  perform¬ 
ance  [3:3.5] . 

The  DRFP  package  follows  the  same  format  of  the  formal 
RFP  package;  however,  the  contracting  officer  has  the  option 
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to  issue  only  those  portions  of  the  package  which  are  complete 
in  order  to  provide  sufficient  time  for  contractor  evaluation 
(1:3.5).  The  DRFP  concept  provides  a  significant  tool  for 
making  solicitation  much  more  effective;  yet  concern  over  its 
application  has  developed.  It  is  this  concern  which  has  ini¬ 
tiated  the  research  effort  to  present  an  exploration  of  the 
DRFP  concept  and  its  effectiveness. 

Literature  Review 

Various  research  has  presented  studies  of  the  acqui¬ 
sitions  process  and  the  formal  RFP  which  are  related  to  the 
DRFP  concept.  Osburn,  in  1973,  completed  a  comprehensive 
model  of  the  Department  of  Defense  procurement  process;  the 
model  showed  the  relationships  between  procurement  management 
functions  and  the  systems  acquisition  environment  (19:6).  His 
findings  included  the  following  observations: 

1)  The  primary  problem  in  preparing  RFPs  is  deter¬ 
mining  the  optimal  balance  between  technical  feasibility  and 
contractual  clarity  in  the  statement  of  work,  specifications, 
and  other  requirements.  This  process  requires  adequate  time 
which  ‘is  not  always  available  in  the  acquisition  process  (19: 
220) . 

2)  There  are  problems  in  defining  requirements  in 
the  RFP  for  contractors  to  use  the  most  cost-effective  materi¬ 
als,  techniques,  and  equipment.  It  appears  to  be  extremely 
difficult  to  prepare  clear  and  unambiguous  specifications  that 
are  needed  in  the  preparations  of  RFPs.  In  some  cases,  the 


requirements  are  not  known  in  enough  detail  to  specify  clearly 
(19:221) . 

3)  Individuals  with  enough  experience  and  expertise 
are  not  always  available.  This  sometimes  results  in  the  pre¬ 
paration  of  an  RFP  which  is  not  properly  tailored  to  the 
system  being  acquired. 

4)  Inadequate  lead  time  to  prepare  the  RFP  usually 
results  in  the  preparation  of  a  deficient  RFP.  The  problem 
centers  around  the  ability  to  estimate  an  adequate  period  of 
time  to  prepare  the  RFP,  in  addition  to  having  enough  time 
before  the  RFP  must  be  released  to  contractors  (19:222). 

In  1975,  Turner  identified  and  compared  policy  and 
procedures  concerning  the  source  selection  process  within  each 
DoD  component.  He  identified  problems  of  complexity  and  size 
of  the  RFPs  and  annotated  Congressional  concern  over  excessive 
requirements  and  requests  for  unnecessary  data  (22:22).  Turner 
discussed  the  principle  of  "checks  and  balances"  which  the 
services  instituted  to  insure  that  RFPs  conform  to  the  program 
objectives  and  solicit  only  minimum  essential  data.  The  ser¬ 
vices  accomplished  this  principle  through  a  system  of  review 
boards  which  offer  an  outside  analysis  by  experienced  senior 
persons  who  question  requirements  and  transfuse  RFP  improve¬ 
ment  among  the  services.  After  looking  at  the  improvements 
to  the  system.  Turner  recommended  giving  attention  to  getting 
industry  more  involved  in  the  requirements  and  RFP  development 
process  (22:35). 

Hynes  presented  a  study  of  the  RFP  from  a  communications 
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perspective.  He  surveyed  contract  administration  officers  of 
the  100  largest  defense  contractors  of  1974  to  gain  knowledge 
of  industry's  perception  of  the  RFP.  Hynes  attempted  to  con¬ 
vey  industry  views  of  the  RFP  to  form  a  basis  for  a  new  look 
at  the  RFP  from  the  standpoint  of  making  it  a  better  method 
of  communication  (14:5).  As  an  effort  for  improving  RFPs 
and  the  procurement  process,  Hynes  suggested  "soliciting" 
recommendations  on  contract  terms  and  RFP  instructions  before 
official  solicitation  (14:27). 

These  research  studies  provide  a  description  of  the 
acquisition  process  and  an  evaluation  of  the  RFP.  An  under¬ 
standing  of  these  concepts  is  necessary  before  attempting  a 
study  of  the  DRFP  concept.  Additionally,  these  studies  pre¬ 
sented  some  of  the  weaknesses  of  the  solicitation  process 
which  the  DRFP  concept  attempts  to  correct.  In  a  study  more 
closely  related  to  the  DRFP,  Allen  and  Hubert  explored  ways 
to  decrease  contractors'  reluctance  to  share  data  which  the 
government  considers  necessary.  They  observed  that  the  proper 
use  of  DRFPs  increased  the  visibility  for  potential  data  prob¬ 
lems  prior  to  formal  solicitation.  Allen  and  Hubert  concluded 
that  this  created  the  opportunity  to  solve  disagreements 
before  initiation  of  a  lengthy  and  costly  negotiation  process 
(6:72).  Their  observations  resulted  from  a  research  effort 
directed  toward  government  ownership  of  source  selection 
sensitive  data;  therefore,  they  did  not  attempt  to  describe 
the  DRFP  concept  or  assess  its  effectiveness.  No  other  infor¬ 
mation  was  discovered  which  related  research  studies  to  the 


DRFP  concept;  however,  additional  information  is  available  in 

* 

regulations  and  documents. 

The  Defense  Acquisition  Regulation  (DAR)  establishes 
the  Department  of  Defense  policies  and  procedures  which  relate 
to  acquisition;  this  includes  guidance  for  the  solicitation 
process  (23:1.1).  Air  Force  Systems  Command  Supplement  to 
the  DAR  contains  the  guidance  for  applying  the  DRFP  concept. 
This  document  includes  identification  of  programs  which  are 
applicable  for  DRFPs  and  emphasis  of  areas  which  have  potential 
for  cost  reduction  from  use  of  DRFPs  (3:3-5).  AFSC  Pamphlet 
70-4,  RFP  Preparation  Guide,  provides  guidelines  for  develop¬ 
ing  and  tailoring  the  RFP;  instructions  for  the  DRFP  are  in¬ 
cluded  in  this  document  (5:8).  Objectives  of  this  document 
include : 

1)  identification  of  key  steps  to  improve  communi¬ 
cation  between  Government  and  industry  and  to 
obtain  more  responsive  proposals, 

2)  simplification  of  the  acquisition  process  and 
reduction  of  paperwork, 

3)  reduction  in  the  size  of  RFPs  while  expressing 
Government  requirements  to  prospective  contrac¬ 
tors  in  clear  terms ,  and 

4)  reduction  in  the  need  for  RFP  modification  [5:2]. 
These  objectives  reflect  the  purpose  of  the  DRFP. 

In  addition  to  the  information  contained  in  these 
documents,  interviews  provided  a  source  of  knowledge  and  guid¬ 
ance  for  the  research  effort.  The  interview  with  Major  Golden 
confirmed  the  need  for  a  research  study  of  the  DRFP  concept. 
Additional  discussion  established  the  objectives  of  the 
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research  effort  (13). 

The  interview  with  Mr.  Carl  Brigner  focused  upon  the 
feasibility  of  a  research  study  of  the  DRFP  concept  (9).  In 
addition,  Mr.  George  Walter,  of  the  Contract  Review  Division, 
Directorate  of  Research  and  Development  Contracting,  assisted 
in  establishing  an  operational  definition  for  the  effective¬ 
ness  of  the  DRFP  concept.  He  suggested  determining  whether 
problem  areas  in  the  solicitation  process  are  identified 
through  the  DRFP.  He  also  prescribed  researching  how  many 
revisions  are  proposed  by  offerors  in  response  to  the  DRFP. 
Finally,  Mr.  Walter  suggested  determining  how  many  recommended 
revisions  are  implemented  in  the  formal  RFP  (15) . 

Research  Objectives/ 

Research  Questions 

This  research  effort  focuses  upon  the  following 
obj  ectives : 

Objective  1.0  -  Describe  the  Draft  Request  for  Proposal  con¬ 
cept.  To  accomplish  this  objective,  the 
research  effort  addresses  the  following 
questions: 

1.1  -  What  are  the  objectives  of  the  Draft 

Request  for  Proposal  concept? 

1.2  -  What  are  the  procedures  for  implementing 

the  Draft  Request  for  Proposal  concept? 

Objective  2.0  -  Assess  the  effectiveness  of  the  Draft  Request 
for  Proposal  concept.  To  accomplish  this 
objective,  the  research  effort  addresses 


the  following  questions: 


2.1  -  How  effective  is  the  Draft  Request  for  Pro¬ 

posal  concept  in  achieving  its  stated  objec¬ 
tives? 

2.1a  -  Does  the  Draft  Request  for  Proposal 
result  in  identification  of  problem 
areas  during  solicitation? 

2.1b  -  How  many  revisions  do  offerors  recom¬ 
mend  in  response  to  the  Draft  Request 
for  Proposal? 

2.1c  -  How  many  recommended  revisions  are 

implemented  in  the  formal  Request  for 
Proposal? 

2. Id  -  Is  there  a  relationship  between  the 
effectiveness  of  the  Draft  Request 
for  Proposal  and  completeness  of  the 
Draft  Request  for  Proposal  package? 

2.2  -  Is  there  a  difference  in  the  procedures  for 

using  the  Draft  Request  for  Proposal  and  re¬ 
sults  of  the  Draft  Request  for  Proposal  in 
different  phases  of  the  acquisition  process? 

Objective  3.0  -  Determine  the  factors  that  decrease  or  in¬ 
crease  the  effectiveness*  of  the  Draft  Request 
for  Proposal  concept  from  the  perspective  of 
government  contracting  officers  and  buyers. 

To  accomplish  this  objective,  the  research 
effort  addresses  the  following  questions: 
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3.1  -  What  factors  decrease  the  effectiveness  of 

the  Draft  Request  for  Proposal  concept? 

3.2  -  What  factors  increase  the  effectiveness  of 

the  Draft  Request  for  Proposal  concept? 
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CHAPTER  II 


METHODOLOGY 

This  chapter  describes  the  sources  of  data  collected, 
the  operational  definition  of  effectiveness,  the  justification 
of  the  data  source,  the  research  structure  used  in  analyzing 
the  data,  and  the  assumptions  and  limitations  of  the  research 
effort . 

Sources  of  Data 

The  sources  of  data  were  government  regulations, 
policy  letters,  reports,  contracts,  and  personal  interviews. 
Contracts  which  included  DRFPs  during  TY  1980  and  1981  pro¬ 
vided  data  on  the  results  of  the  DRFP.  The  personal  interviews 
solicited  knowledge  from  Principal  Contracting  Officers  (PCO) 
and  buyers  who  had  actually  used  DRFPs. 


Operational  Definition  of 
Effectiveness 

An  effective  DRFP  is  one  that  results  in  implementa¬ 
tions  which  address  the  areas  of  potential  cost  savings. 

These  areas  include: 

1)  Changes  to  requirements  specifying  manufacturing 
processes,  facilities,  tools,  and  test  capabili¬ 
ties  , 

2)  Schedule  or  delivery  changes  that  will  result  in 
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cost  reductions, 

3)  Substitution  of  RFP-stated  materials;  and  overly 
restrictive  engineering  and  quality  specifications 
and  requirements , 

4)  The  use  of  common  government- furnished  equipment 
or  standard  commercial  components  instead  of 
system  peculiar  items, 

5)  Reduction,  substitution,  or  deletion  of  military 
specifications  and  standards, 

6)  Excessive  reporting  documents, 

7)  Modified  contractual  arrangements  designed  to 
financially  incentivize  contractors  and  subcon¬ 
tractors  to  pursue  cost-reduction  efforts  during 
contract  performance. 

The  purpose  of  this  definition  is  to  distinguish  signficant 
contractor  recommendations  which  facilitate  the  purpose  of 
the  DRFP  concept- -solicitation  of  industry  feedback  that  may 
provide  significant  cost  savings  and  program  improvements. 

Data  Source 

Research  for  this  study  was  conducted  within  Aero¬ 
nautical  Systems  Division  (ASD) ,  Wright-Patterson  AFB  (WPAFB) 
Ohio.  ASD  responsibilities  include  managing  "the  acquisition 
of  aeronautical  systems,  subsystems,  and  related  equipment 
programs  and  projects.  .  .  [4:1]."  During  FY  1980,  ASD  was 
responsible  for  negotiating  44  percent  of  AFSC  contractual 


actions, 1  or  7,866  of  17,979.  This  involved  $8.38  billion  of 
$12,948  billion  of  obligations  in  AFSC ,  or  65  percent  (1). 
Three  organizations  within  ASD  provided  data  for  the  research: 
Directorate  of  Research  and  Development  Contracting  (PMR) , 
Deputy  for  Reconnaissance  and  Electronic  Warfare  Systems  (RW) , 
and  Deputy  for  Aeronautical  Equipment  (AE) .  PMR  is  oriented 
toward  the  acquisition  of  research  and  development,  support 
services,  and  specialized  equipment  for  Air  Force  laboratories 
at  WPAFB,  Aerospace  Medical  Division  at  Brooks  AFB,  Air  Force 
Human  Resources  Laboratory  at  Brooks  AFB,  and  the  Air  Force 
Business  Research  Management  Center  (2) .  PMR  completed  53 
percent  of  ASD  contracts  in  FY  1980;  this  accounted  for  $361 
million  in  obligations  (1).  RW  is  responsible  for  reconnais¬ 
sance  and  electronic  warfare  systems  in  support  of  Department 
of  Defense  missions.  During  FY  1980,  RW  negotiated  8  percent 
of  ASD  contracts  which  represented  6  percent  of  total  obliga¬ 
tions.  AE  accomplishes  the  acquisition  of  avionics  and  air¬ 
craft  accessories,  life  support,  and  support  equipment.  AE 
completed  8  percent  of  ASD  contracts  in  FY  1980  for  2  percent 
of  obligations  (1). 

Research  Structure 

Research  Questions  1.1/1. 2 

What  are  the  objectives  of  the  DRFP  concept? 

What  are  the  procedures  for  implementing  the  DRFP  concept? 

*For  purposes  of  this  discussion,  contractual  actions 
are  considered  to  include  new  contracts,  supplemental  agree¬ 
ments,  and  modifications. 
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An  extensive  review  of  defense  documents  provided 
data  to  determine  the  objectives  and  procedures  of  the  DRFP 
concept.  Documents  which  address  the  DRFP  include: 

AFSC  Defense  Acquisition  Regulation  Supplement 
AFSC  Pamphlet  70-4:  RFP  Preparation  Guide 
AFSC  and  ASD  policy  letters 

A  subjective  evaluation  of  the  data  from  these  sources  fur¬ 
nished  the  objectives  and  procedures  of  the  DRFP  concept. 
Research  Question  2.1 

How  effective  is  the  DRFP  concept  in  achieving  its  stated 
obj  ective? 

Data  collected  from  contracts  which  incorporated  the 
DRFP  were  used  to  describe  the  effectiveness  of  the  DRFP. 
Criteria  for  selecting  a  sample  of  contracts  included: 

1)  contracts  which  have  used  the  DRFP, 

2)  contracts  which  have  completed  the  procurement 
planning  phase  of  the  contracting  process,  such 
that  a  RFP  has  been  developed  which  incorporates 
responses  to  the  DRFP, 

3)  contracts  which  have  not  changed  in  scope  or 
objective  during  the  contracting  process,  such 
that  a  revised  DRFP  was  not  issued  to  industry,  and 

4)  contracts  which  represent  a  wide  spectrum  of  dollar 
amounts  based  upon  the  government  estimate  of 
contract  cost. 

Initial  planning  included  a  random  sample  of  the  con¬ 
tracts  for  data  collection.  Preliminary  inquiries  resulted 
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in  an  expected  population  of  about  40  contracts  in  PMR  and 
between  12  to  15  contracts  in  AE  and  RW.  To  allow  sufficient 
time  for  incorporation  of  recommendations  in  the  formal  RFP, 
a  cutoff  date  of  30  September  1980  was  used  for  issuance  of 
the  DRFPs.  The  cutoff  date  limited  the  number  of  DRFPs  to  a 
population  of  31  in  PMR.  From  this  source,  a  random  selection 
provided  12  contracts  for  analysis  of  the  DRFPs.  The  number 
of  the  sample  DRFPs  was  not  determined  statistically  due  to 
the  nature  of  the  research  and  an  undefined  standard  deviation. 
However,  the  random  selection,  in  the  judgment  of  the  research¬ 
ers,  provided  a  cross-section  of  contracts  in  PMR. 

Verification  of  the  population  in  AE  and  RW  resulted 
in  a  decrease  to  10  contracts.  The  reason  for  the  reduction 
was  program  cancellation  prior  to  issuance  of  a  formal  RFP. 
Thus,  the  methodology  consisted  of  a  census  of  these  contracts 
due  to  the  low  number.  Thereafter,  the  data  collection  yielded 
only  five  contracts  for  analysis  of  the  DRFP.  The  other  five 
contracts  were  disqualified  due  to  either  the  contracting  pro¬ 
cess  having  not  progressed  to  incorporation  of  recommendations 
into  the  formal  RFP  or  the  DRFP  being  issued  concurrently  as 
the  formal  RFP.  The  small  data  source  provided  excellent 
examples  of  the  DRFP  application,  but  it  has  limited  the 
ability  to  generalize  the  subsequent  findings. 

A  structured  research  guide  controlled  the  data 
collection  to  insure  a  systematic  evaluation  of  each  DRFP 
(see  Appendix  A).  Specific  collection  efforts  included: 

1)  delineating  DRFP  in  terms  of  forecasted  dollar 


amount  and  suggested  contract  type, 

2)  documenting  contractor  responses  which  address 
general  problem  areas  and  do  not  provide  specific 
recommendations , 

3)  identify  number  of  recommendations  from  contrac¬ 
tors  and  number  of  revisions  incorporated  in  RFP 
according  to  section  of  DRFP  which  changes  are 
applicable  to,  and 

4)  differentiating  between  recommendations  which  meet 
the  operational  definition  of  effectiveness  and 
those  which  do  not  meet  the  definition. 

Contracting  officers  of  PMR,  AE,  and  RW  validated 
the  research  guide  by  reviewing  it  and  offering  suggestions 
for  improvement.  This  validation  provided  the  opportunity  to 
evaluate  the  feasibility  of  the  guide  for  analyzing  DRFPs.  As 
a  result  of  the  review,  two  additional  elements  were  added  to 
the  operational  definition  of  effectiveness  for  the  data  collec¬ 
tion.  These  elements  were:  1)  identification,  simplification, 
or  reduction  of  special  clauses,  and  2)  compatibility  of  con¬ 
tract  type  with  statement  of  work. 

Analysis  of  data  provided  a  description  of  the  effec¬ 
tiveness  of  the  DRFP  according  to  the  number  of  changes  which 
meet  the  criteria  of  the  operational  definition  of  effective¬ 
ness.  Additionally,  the  analysis  attempted  to  describe  the 
relationship  between  effectiveness  and  the  degree  of  complete¬ 
ness  of  the  DRFP  package,  based  upon  the  number  of  completed 
sections  of  the  DRFP.  The  statistical  tool,  regression 
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analysis,  was  used  to  analyze  the  relationship  between  the 
number  of  changes  and  the  number  of  sections  completed. 

Regression  analysis  provides  a  method  to  examine  the 
relation  between  a  variable  of  interest  and  one  or  more  inde¬ 
pendent  or  predictor  variables  (16:434).  The  equation 

y  -  e„  *  stx 

represents  the  estimated  regression  function  of  the  simple 
linear  regression  model.  The  variable,  Y,  represents  the  num¬ 
ber  of  changes  and  the  variable,  X,  corresponds  to  the  number 
of  completed  sections  of  the  DRFP  package.  The  point  esti¬ 
mator  of  6^,  b^,  represents  the  relative  importance  of  the 
independent  variable.  For  example,  as  the  number  of  completed 
sections  increases,  the  number  of  changes  are  expected  to  in¬ 
crease  by  the  value  of  b^.  The  Statistical  Package  for  Social 
Sciences  (SPSS)  provides  a  computer  program  for  deriving 
estimates  of  b^.  Another  output  of  the  SPSS  computer  program 
is  the  coefficient  of  simple  correlation,  r,  which  is  a  measure 
of  the  degree  of  linear  relationship  between  the  independent 
and  dependent  variables  (17:276).  The  closer  that  the  absol¬ 
ute  value  of  r  is  to  the  value  of  one,  the  greater  the  degree 
of  relationship  between  the  variables. 

In  addition  to  using  the  coefficient  of  simple  cor¬ 
relation  to  describe  the  strength  of  the  relationship,  the 
research  also  tested  whether  Sj  -  0.  If  *  0,  then  the  ex¬ 
pected  value  of  Y  would  equal  0Q  ♦  0jX  ■  0Q  for  all  values  of 
X.  Thus,  all  values  of  Y  would  have  the  same  mean  and,  by 
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implication,  all  distributions  would  be  identical  and  there 
would  be  no  significant  relationship  between  X  and  Y.  The 
appropriate  hypotheses  for  this  test  are: 

H0:  81  =  0 
H1:  *  0 

The  first  test  of  these  hypotheses  involved  con¬ 
structing  a  confidence  interval  for  g^  based  upon  the  theorem: 

bi  -  $i 

sTEjf  ’  t("-2) 

In  accordance  with  this  theorem,  the  confidence  interval  takes 
the  form  of  L  <  <  U  where: 

L  =  bx  -  t(l-a/2;n-2)s(b1) 

U  »  bx  +  t(l-a/2;n-2)s(b1) 

A  90  percent  confidence  level  was  established  for  testing  the 
hypotheses  since,  even  if  it  were  possible  to  obtain  all 
available  information,  the  decision  cannot  be  made  with  abso¬ 
lute  certainty  but,  rather,  must  be  based  on  probabilities 
(16:474;  17:268). 

A  second  test  to  confirm  that  g^  ■  0  involved  the 
standardized  test  statistic,  t* ,  where: 

bj-0  bx 

'*  *  ’  57^ 

The  appropriate  decision  rule  for  t*  is: 

If  |t*|  <_  t(l-a/2;n-2) ,  conclude  HQ 
If  |t*|  >  t(l-a/2 ;n-2) ,  conclude  ^ 
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The  final  test  of  involved  a  comparison  of 
with  action  limits.  The  formulae  for  the  action  limits  are: 

=  0  +  t(a/2 ; n - 2 ) s (b^) 

A2  *  0  +  t(l-a/2 ;n-2) s(b1) 

The  appropriate  decision  rule  for  the  comparison  of  bj^  with 
action  limits  is: 

If  Aj  £  b^  <  »  conclude  Hq 

If  b^  <  A^  or  b^  >  A2,  conclude 

After  analyzing  the  relationship  between  the  number 
of  changes  and  the  number  of  sections,  the  proposed  dollar 
amount  was  substituted  as  the  independent  variable.  This  sub¬ 
stitution  was  accomplished  to  evaluate  the  relationship  between 
the  number  of  changes  and  the  dollar  amount  of  the  contracts. 

The  analysis  consisted  of  the  same  tests  which  were  conducted 

for  the  variable,  number  of  sections. 

Research  Question  2.2 

Is  there  a  difference  in  the  procedures  for  using  the 
DRFP  and  the  results  of  the  DRFP  in  the  different 
phases  of  the  acquisition  process? 

The  research  effort  analyzed  data  from  personal  inter¬ 
views  to  compare  procedures  and  actual  practices  in  PMR  against 
AE  and  RW.  This  provided  a  subjective  appraisal  of  differences 
in  procedures  for  using  the  DRFP  in  different  phases  of  the 
acquisition  process  since  the  efforts  of  PMR  focus  upon  the 
initial  phases  and  the  efforts  of  AE  and  RW  concentrate  upon 
the  Full-Scale  Development  through  Production  and  Deployment 
phases. 


To  distinguish  results  of  the  DRFP,  the  mean  number 
of  revisions  in  PMR  contracts  was  compared  against  the  mean 
number  of  revisions  in  AE  and  RW  contracts.  The  T-Test  pro¬ 
vided  the  statistical  tool  for  testing  the  difference  between 
means.  The  first  step  in  using  the  T-Test  was  establishing 
a  null  and  alternative  hypothesis.  For  this  study,  the 
hypotheses  were: 

Hq:  ‘  y2  =  ®  (no  difference  in  results) 

**  ®  (difference  exists) 

Again,  a  90  percent  confidence  level  was  used  for  testing  the 
hypotheses.  Since  a  90  percent  confidence  level  was  estab¬ 
lished,  an  a  value  of  .10  was  used. 

The  SPSS  subprogram,  T-Test,  computed  the  test  sta¬ 
tistics.  Formulae  required  for  manual  calculations  are  pre¬ 
sented  in  Appendix  B  for  reference.  The  variables  of  interest 
for  the  T-Test  were  the  source  of  the  DRFP  and  the  number  of 
changes  which  met  the  operational  definition  of  effectiveness. 

The  first  test  of  the  hypotheses  consisted  of  estab¬ 
lishing  a  confidence  interval  for  y2  "  Uj  (16:313).  The  form 
used  was  L  <_  y2  "  wi  iu  where: 

L  -  D  -  t(l-a/2;n1+n2-2)s(D) 

U  -  D  +  t(l-o/2;n1+n2-2)s(D) 

The  second  test  was  a  comparison  of  the  probability 
value  with  the  a  value  of  .10.  This  test  involves  the  deci¬ 


sion  rule: 


If  the  probability  value  is  greater  than  a  , 
conclude  Hq 

If  the  probability  value  is  less  than  a  , 
conclude 

The  next  test  involved  the  calculation  of  action 
limits  for  comparison  with  the  value  of  y2  "  Uj*  The  action 
limits  are  based  upon  the  t  distribution  for  a/2  and  l-a/2. 
The  formulae  for  the  action  limits  are: 

Ax  =  0  -  t(l-a/2;n1+n2-2)s(D) 

A2  *  0  +  t(l-a/2  ;n^+n2*2)  s(ir) 

The  decision  rule  was:’ 

If  A ^  <_  D  <_  A2>  conclude  Hq 

If  D  <  or  U  >  A2,  conclude 

The  fourth  test  compared  the  computed  value  of  the 
test  statistic  with  the  critical  value  of  t.  The  critical 
value  of  t  was  determined  from  the  statistical  table  of  the 
t  distribution  where: 


Critical  =  t(l-a/2;n1+n2-2) 

The  appropriate  decision  rule  was: 

If  ‘compute  i  ‘critical'  ""elude  HQ 
If  ‘compute  >  ‘critical'  conclude  Hj 
Research  Questions  3. 1/3. 2 

What  factors  decrease  the  effectiveness  of  the  DRFP 
concept? 

What  factors  increase  the  effectiveness  of  the  DRFP 
concept? 
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A  structured  interview  directed  to  PCOs  and  buyers 
provided  data  for  determining  factors  which  decrease  and  in¬ 
crease  the  effectiveness  of  the  DRFP  concept.  Data  from 
literature  review  and  analysis  of  DRFPs  provided  questions 
for  the  interview.  General  areas  of  questions  addressed  the 
preparation  of  the  DRFP  and  the  factors  which  influence  re- 
i  suits.  The  greatest  advantages  of  the  interview  are  the 
depth  and  detail  of  information  which  can  be  obtained.  Also, 
the  interviewer  has  more  control  over  the  process  than  other 
methods  (11:294).  With  this  method,  information  of  factors 
which  influence  the  DRFP  concept  can  be  obtained  which, 
otherwise,  might  be  undetected  by  literature  review  and 
analysis  of  contracts. 

One  major  limitation  of  personal  interviews  is  bias. 
An  interview  schedule  which  consisted  of  a  set  of  structured 
questions  provided  a  method  to  control  bias  (see  Appendix  C) . 
Evaluation  by  contracting  and  acquisition  management  instruc¬ 
tors  from  the  Air  Force  Institute  of  Technology  (AFIT)  pro¬ 
vided  a  method  to  validate  the  interview  schedule.  This 
process  identified  weaknesses  in'  the  interview  schedule  and 
provided  suggestions  for  improvement.  The  main  result  of  the 
evaluation  was  a  rewording  of  several  questions  to  avoid  any 
impression  of  negative  bias  during  the  interviews.  The  AFIT 
faculty  comprised  a  valid  testing  source  since  all  were 
former  contracting  officers  or  had  experience  and  knowledge 
of  contracting  procedures  in  the  acquisition  environment. 

Twelve  interviews  were  conducted  with  Principal 
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Contracting  Officers  and  buyers  from  PMR,  AE  and  RW.  Inter¬ 
viewees  selected  were  the  individuals  responsible  for  the 
issuance  of  the  DRFPs  included  in  the  data  source.  Five 
individuals  were  not  available  for  the  interviews  due  to 
their  relocation  or  retirement.  All  interviews  were  recorded 
on  tape  and  afterwards  transcribed  in  their  entirety.  This 
method  allowed  full  attention  to  the  interviewees  and  also 
insured  a  complete  record  of  the  responses. 

Assumptions 

1.  The  responses  expressed  in  interviews  were  true 
perceptions  based  upon  specific  experience  with  DRFPs. 

2.  The  sample  of  contracts  was  representative  of 
contracts  in  ASD  which  incorporated  DRFPs. 

3.  The  populations  from  which  the  sample  of  con¬ 
tracts  were  selected  were  normal  or  the  departures  were  not 
too  significant,  and  the  populations  had  the  same  variance. 
These  assumptions  were  required  for  the  application  of  the  T- 
Test. 

4.  In  the  application  of  regression  analysis,  the 
error  terms  were  independent  and  normally  distributed  with 
an  expected  value  of  zero  and  constant  variance. 

Limitations 

1.  Research  effort  was  confined  to  ASD  due  to 
limited  time  and  available  funds. 

2.  Due  to  the  time  constraint,  the  research  effort 
was  limited  to  the  PMR,  AE,  and  RW  directorates  of  ASD. 
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3.  Results  of  this  research  were  not  generalizable 
beyond  ASD  since  this  organization  is  responsible  for  a  mission 
unique  to  the  rest  of  AFSC. 

4.  Some  subjectivity  was  exercised  by  the  research 
team  in  evaluating  and  categorizing  contractor  responses  to 
DRFPs . 

The  following  chapter  presents  the  summary  and  analy¬ 
sis  of  data  which  have  been  collected  under  the  research 
methodology  presented  in  this  chapter. 
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CHAPTER  III 

DATA  SUMMARY  AND  ANALYSIS 

This  chapter  presents  the  data  collected  from  review 
of  defense  documents,  analysis  of  contracts,  and  interviews 
with  contracting  officers  and  buyers.  AFSC  supplement  to 
DAR,  AFSC  Pamphlet  70-4,  and  policy  letters  provided  the 
stated  objectives  and  procedures  for  the  DRFP  concept.  The 
analysis  of  contract  files  afforded  knowledge  of  the  actual 
effectiveness  and  procedures  for  using  the  DRFP.  Finally, 
the  personal  interviews  furnished  insight  into  the  factors 
which  decrease  or  increase  the  effectiveness  of  the  DRFP 
concept . . 

Review  of  Defense  Documents 

Objectives .  In  answering  Research  Questions  1.1/1. 2, 
the  review  of  defense  documents  provided  the  objectives  and 
procedures  for  implementing  the  DRFP  concept.  The  purpose 
of  the  DRFP  concept  is  to  provide  significant  cost  savings 
and  improvements  in  planned  acquisitions  through  industry 
feedback.  In  support  of  this  purpose,  the  DRFP  attempts  to 
provide  industry  with  greater  insight  and  appreciation  of  re¬ 
quirements,  opportunity  to  offer  comments  and  suggestions  for 
refining  requirements,  and  consequently  result  in  more  defined 
and  realistic  proposals.  Since  the  Air  Force  does  not  always 
know;  exactly  how  to  specify  needs,  the  specific  objectives  of 
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the  DRFP  are  to  identify: 

1)  Changes  to  requirements  specifying  manufacturing 
processes,  facilities,  tools,  and  test  capabili¬ 
ties  , 

2)  Schedule  or  delivery  changes  that  will  result  in 
cost  reductions, 

3)  Substitution  of  RFP- stated  materials;  and  overly 
restrictive  engineering  and  quality  specifications 
and  requirements, 

4)  The  use  of  common  government- furnished  equipment 
or  standard  commercial  components  instead  of 
system  peculiar  items, 

5)  Reduction,  substitution,  or  deletion  of  military 
specifications  and  standards, 

6)  Excessive  reporting  requirements  (to  include  data 
requirements) , 

7)  Modified  contractual  arrangements  designed  to  fin¬ 
ancially  incentivize  contractors  and  subcontractors 
to  pursue  cost-reduction  efforts  during  contract 
performance. 

Procedures .  DRFPs  are  required  for  all  R§D  contracts 
estimated  to  exceed  $500,000  and  all  other  contracts  over 
$1  million,  except  contracts  for  sole  source  which  are  follow- 
on  contracts  for  the  same  item  from  the  same  source.  The 
activities  for  issuing  DRFPs  begin  with  early  planning  in  the 
contracting  process.  For  acquisitions  requiring  a  Secretarial 
Determination  and  Finding  (D§F) ,  the  DRFP  can  be  a  parallel 
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activity  to  the  processing  of  the  D§F.  Additionally,  the  DRFP 
can  be  issued  prior  to  approval  of  the  D§F,  but  negotiations 
will  not  be  conducted  until  approval  of  the  D§F  and  issuance 
of  a  formal  RFP.  The  Air  Force  will  synopsize  the  DRFP  in  the 
Commerce  Business  Daily  unless  exempted  by  DAR.  Contracting 
personnel  will  expedite  or  reduce  internal  reviews  to  permit 
maximum  time  for  contractor  review  and  then  issue  the  DRFP 
to  all  sources  which  are  furnished  by  the  project  engineer. 
Additionally,  the  contracting  officer  will  issue  a  copy  of  the 
DRFP  to  the  cognizant  contract  administration  office  of  a 
potential  offeror  when  the  DRFP  is  issued  to  industry.  This 
requirement  provides  the  opportunity  for  the  contract  admini¬ 
stration  offices  to  give  early  assistance  in  identifying  and 
eliminating  potential  problems  in  proposals. 

Requirements  for  the  content  of  the  DRFP  allow  for 
issuance  of  either  a  complete  package  or  selected  sections  of 
the  RFP  package.  Minimum  inputs  include  instructions,  evalua¬ 
tion  factors,  special  and  general  provisions,  description  or 
specifications,  and  data  lists.  Contracting  officers  must 
also  include  an  executive  summary  letter  which  accomplishes 
at  least  the  following  tasks: 

1)  description  of  the  DRFP  objectives  and  applicable 
procedures , 

2)  encouragement  to  evaluate  and  challenge  any  ele¬ 
ments  of  the  DRFP  package, 

3)  emphasis  that  responses  are  voluntary  and  will 
remain  anonymous  and  that  the  actions  do  not  call 
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for  negotiation  or  offers  during  the  DRFP  process, 

4)  identification  of  high  cost  drivers  and  improve¬ 
ments  for  program  accomplishment,  and 

5)  identification  of  a  general  funding  estimate, 
when  appropriate,  if  competition  is  involved. 

For  major  acquisitions,  a  panel  will  both  review  the 
DRFP  and  assess  the  offerors'  responses  for  implementation 
into  the  formal  RFP.  For  less- than-major  programs,  the  pro¬ 
ject  engineer  will  review  the  responses.  Contracting  officers 
will  acknowledge  contractor  recommendations  either  by  cor¬ 
respondence  or  through  a  non- attribution  conference. 

Analysis  of  Contract  Files 

Presentation  of  data  from  the  contract  files  follows 
the  format  of  the  Research  Guide  and  Worksheet  (Appendix  A) . 
The  collected  data  provided  the  information  necessary  to  des¬ 
cribe  the  effectiveness  of  the  DRFP  in  response  to  Research 
Question  2.1.  The  contracts  also  provided  the  necessary  data 
to  compare  results  of  the  DRFP  during  different  phases  of  the 
acquisition  process  to  answer  Research  Question  2.2.  Speci¬ 
fically,  the  data  provided  the  necessary  information  to 
describe  the  relationship  between  the  DRFP's  effectiveness 
and  degree  of  completion  of  the  DRFP  package.  Finally,  the 
collection  provided  data  to  compare  the  results  of  the  DRFP 
in  PMR  vis-a-vis  AE  and  RW. 

Estimated  Dollar  Amount.  Table  1  presents  the  dis¬ 
tribution  of  contracts  in  terms  of  estimated  dollar  amount. 


TABLE  1 


Estimated  Dollar  Amounts  of  Contracts 


Dollar  Amount 

Number 

of  Contracts 

PMR 

0  -  500,000 

1 

500,000  -  1,000,000 

5 

1,000,000  -  1,500,000 

3 

1,500,000  -  3,500,000 

3 

AE/RW 

0  -  1,000,000 

0 

1,000,000  -  1,500,000 

2 

1,500,000  -  3,500,000 

0 

3,500,000  -10,000,000 

1 

10,000,000  -20,000,000 

1 

over  20,000,000  . 

1» 

♦Estimated  dollar  amount  was 

$155,000,000 

This  data  was  extracted  from  the  Determination  and  Findings 
for  each  contract.  In  PMR,  the  values  ranged  from  $390,000  to 
$3,500,000,  with  the  largest  number  of  contracts  falling 
between  $500,000  to  $1,000,00Q.  In  AE  and  RW,  the  dollar 
amounts  varied  from  $1,400,000  to  $155,000,000;  there  was 
approximately  a  uniform  distribution  of  these  contracts  across 
the  dollar  values.  The  distributions  of  contracts  in  terms 
of  dollar  amounts  describes  the  DRFPs  which  provided  the  data 
base  for  the  research. 

Type  of  Contract.  This  data  also  describes  the  source 
of  the  data  base.  As  illustrated  in  Table  2,  the  contracts 
were  both  fixed  price  and  cost  reimbursement  types.  In  PMR, 

58  percent  were  Cost  Plus  Fixed  Fee  contracts.  No  one  type 


TABLE  2 


Type  of  Contracts 


Type  of  Contract 

Number 

PMR 

Fixed  Price  (Level  of  Effort) 

1 

Cost  Plus  Fixed  Fee 

7 

Firm  Fixed  Price 

4 

AE/RW 

Cost  Plus  Incentive  Fee 

2 

Fixed  Price  Incentive 

1 

Firm  Fixed  Price 

2 

dominated  in  AE  and  RW  as  the  contracts  ranged  from  Cost  Plus 
Incentive  Fee  to  Firm  Fixed  Price. 

Number  of  Responses  Solicited.  This  section  describes 
the  scope  of  the  industry  contractors  who  received  DRFPs  from 
the  data  base.  Of  the  twelve  contracts,  PMR  issued  DRFPs  to 
twenty  or  fewer  contractors  in  eleven  cases.  Solicitations 
per  contract  ranged  from  three  to  a  maximum  of  73.  Three  of 
the  five  AE/RW  contracts  involved  DRFPs  being  sent  to  ten  or 
fewer  contractors;  the  number  of  contractors  solicited  ranged 
from  six  to  59  for  the  five  contracts.  Table  3  illustrates 
the  distribution  of  contracts  in  terms  of  number  of  responses 
solicited. 

Number  of  Responses  Received.  In  comparison  with  the 
number  of  contractors  solicited,  this  section  describes  the 
responsiveness  of  contractors  in  terms  of  the  number  who  res¬ 
ponded  to  the  DRFPs.  The  values  represent  contractors  who 
actually  provided  feedback  to  the  DRFP  and  does  not  include 


TABLE  3 


Number  of  Responses  Solicited 


Number  of  Responses 

Number  of  Contracts 

PMR 

1  - 

10 

5 

11  - 

20 

6 

21  - 

« 

30 

0 

• 

71  - 

80 

1 

AE/RW 

1  - 

10 

3 

11  - 

20 

0 

21  - 

• 

30 

1 

• 

51  - 

60 

1 

TABLE  4 

Number  of  Responses  Received 


those  who  only  acknowledged  its  receipt.  Table  4  presents 
the  distribution  of  contracts  in  this  section.  Of  the  twelve 
PMR  contracts,  one-third  resulted  in  only  one  response  to  the 


DFRP;  the  maximum  number  of  responses  was  eight.  The  number 
of  responses  for  AE  and  RW  contracts  ranged  from  three  to 
eight . 

Sections  Included  in  the  DRFP.  This  area  presents 
the  number  of  sections  of  the  RFP  package  which  were  included 
in  the  DRFP.  The  data  not  only  describes  the  completeness  of 
the  DRFP  package,  but  also  distinguishes  the  sections  which 
were  most  common.  Table  5  presents  the  distributions  of  PMR 
and  AE/RW  contracts.  The  description/specifications,  or 
statement  of  work  (SOW),  appeared  in  100  percent  of  all  DRFPs. 
The  Contract  Data  Requirements  List  (CDRL)  was  another  domi¬ 
nant  item,  common  in  16  of  the  DRFPs.  The  two  sections, 
"Instructions  and  Conditions,  and  Notices  to  Offerors,"  and 
"Evaluation  Factors  for  Award"  appeared  in  50  percent  of  the 
PMR  DRFPs.  Other  sections  of  the  PMR  DRFPs  were  "General 
Provisions"  and  "Special  Provisions,"  which  appeared  once. 

In  addition  to  the  SOW  and  CDRL,  other  sections  and  number 
of  applications  in  AE/RW  contracts  were  "Contract  Form"--l, 
"Deliveries  or  Performance"- -1,  and  "Evaluation  Factors  for 
Award"- -2. 

Contractor  Responses  to  General  Problem  Areas.  The 
primary  interest  in  researching  contractor  responses  was  to 
evaluate  the  effectiveness  of  DRFPs  in  terms  of  contractor- 
suggested  changes  to  the  formal  RFP.  The  operational  defi¬ 
nition  of  effectiveness  captured  these  recommendations  into 
one  category;  however,  the  research  also  documented  responses 
which  addressed  general  problem  areas  other  than  the  specific 

36 


TABLE  5 


Number  of  Sections  in  DRFP 


Number 

of  Sections 

Number  of  Contracts 

PMR 

1 

0 

2 

3 

3 

5 

4 

3 

5 

1 

AE/RW 

1 

1 

2 

1 

3 

2 

4 

1 

areas  of  the  operational  definition.  Finally,  the  research 
annotated  recommendations  of  an  editorial  nature. 

This  section  presents  the  data  collected  in  the 
category  of  responses  to  general  problem  areas.  Table  6  lists 
the  DRFPs  with  the  number  of  contractor  recommendations  and 
the  number  of  recommendations  which  were  incorporated  in  the 
RFP.  Approximately  32  percent  of  the  contractors*  recommenda¬ 
tions  to  the  12  PMR  DRFPs  resulted  in  changes.  Isolating  the 
AE/RW  DRFPs,  the  percent  of  recommendations  which  resulted  in 
changes  to  the  RFP  was  23  percent.  Overall,  more  than  50  per¬ 
cent  of  these  changes  focused  upon  the  need  to  clarify  or 
define  terms  and  requirements.  Other  responses  highlighted 
missing  data  items,  identified  additional  factors  which  merited 
consideration,  and  revised  percentages  of  effort  required  in 


TABLE  6 


Responses  to  General  Problem  Areas 


DRFP 

Recommendations 

Changes 

PMR 

1 

4 

4 

2 

4 

0 

3 

13 

3 

4 

1 

0 

5 

3 

0 

6 

3 

1 

7 

3 

0 

8 

16 

8 

9 

3 

0 

10 

1 

1 

11 

0 

0 

12 

9 

2 

60 

19 

AE/RW 

1 

28 

5 

2 

6 

0 

3 

17 

S 

4 

2 

1 

5 

_ 7_ 

3 

60 

14 

120 

33 

different  phases  of  work.  Typical  responses  in  this  category 
included: 

--data  missing  from  Figure  2;  leads  to  confusion  over 
requirements , 

--define  and  pTice  Depot  SE  for  equitable  comparison  of 
Life  Cycle  Cost, 

--specify  aircraft  application, 

--Figure  6  conflicts  with  paragraph  4.2.3. 2, 

--CDRL  missing  for  .  .  .  , 

--clarify  whether  one-place  or  two-place  system  to  be  tested. 
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--clarify  requirements  for  monitoring,  and 
--indicate  modes  of  operation  in  paragraph  3.2. 

Contractor  Recommendations  Which  Meet  the  Operational 
Definition.  As  discussed  in  the  previous  section,  this  cate¬ 
gory  was  the  primary  interest  of  the  research.  The  responses 
grouped  in  this  section  met  one  of  the  requirements  of  the 
operational  definition  in  Appendix  A.  The  most  common  elements 
which  the  total  of  48  recommendations  addressed  were:  1) 
changes  to  requirements  specifying  manufacturing  processes, 
facilities,  tools,  and  test  capabilities;  2)  schedule  or  deli¬ 
very  changes  that  will  result  in  cost  reductions;  and  3)  sub¬ 
stitution  of  RFP  stated  materials  and  overly  restrictive 
engineering  and  quality  specifications  and  requirements. 
Elements  which  the  recommendations  did  not  address  were: 

1)  the  use  of  common  government -furnished  equipment  or  stand¬ 
ard  commercial  components  instead  of  system  peculiar  items; 

2)  modified  contractual  arrangements  designed  to  financially 
incentivize  contractors  and  subcontractors  to  pursue  cost- 
reduction  efforts  during  contract  performance;  and  3)  identi¬ 
fication,  simplification,  or  reduction  of  special  clauses. 

Table  7  presents  the  data  applicable  to  this  section.  Thirty- 
five  percent  of  the  contractors'  recommendations  resulted  in 
changes  to  the  PMR  drafts;  the  percentage  for  AE/RW  drafts  was 
18  percent.  The  following  are  examples  of  recommendations 
which  were  implemented: 

--recommend  use  of  1.75  inch  width  versus  2.25  inches, 
--increasing  to  750  p.s.i.  will  affect  design, 
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TABLE  7 


Contractor  Recommendations  Which  Meet 
Operational  Definition 


DRFP 

Recommendations 

Changes 

PMR 

1 

10 

10 

2 

0 

0 

3 

4 

0 

4 

2 

0 

5 

8 

1 

6 

1 

1 

7 

17 

6 

8 

15 

4 

9 

5 

0 

10 

0 

0 

11 

2 

0 

12 

5 

2 

■  — 

— 

69 

24 

AE/RW 

1 

44 

5 

2 

6 

1 

3 

18 

3 

4 

45 

12 

5 

21 

3 

-  - 

134 

24 

2TT 

4S~ 

relax  mandatory  height  to  maximum  of  7.875  inches, 
defer  manufacturing/production  management  system  to 
Phase  III, 

delete  present  thickness  criteria  (adds  to  complexity) , 
reduce  engineering  data  requirements, 
delete  Military  Standard  ...» 

change  from  Firm  Fixed  Price  contract  to  cost-type 
to  make  arrangement  commensurate  with  work  (this 


1 


recommendation  appeared  in  six  of  eight  responses) , 
--restrict  computer  software  to  that  originated  during 
contract , 

--develop  two  programs  and  allow  proposals  to  either 

since  present  program  addresses  parallel  development  of 
two  different  technologies  (recommended  by  five  of  six 
responses) , 

--delete  requirement  for  metallic  thermostructure,  and 
--reduce  requirement  from  100  to  25  samples. 

Contractor  Recommendations  of  an  Editorial  Nature. 

The  final  category  captured  recommendations  which  addressed 
such  items  as  typographical  errors  and  incorrect  references. 

The  purpose  of  collecting  data  in  this  area  was  to  assess  how 
much  effort  is  devoted  by  contractors  on  insignificant  elements 
of  the  DRFP,  and  thus  resulting  in  ineffective  feedback.  In 
other  words,  it  was  felt  that  the  buying  organization  has  the 
capability  to  identify  and  correct  such  errors.  Table  8  de¬ 
picts  the  number  of  recommendations  and  changes  for  each  DRFP. 
This  area  resulted  in  the  fewest  recommendations  and  changes 
for  both  PMR  and  AE/RW.  The  percentage  implemented  by  PMR  was 
25  percent,  compared  to  a  48  percent  implementation  rate  by 
AE  and  RW.  Additionally,  one-third  of  the  PMR  drafts  and  40 
percent  of  the  AE/RW  drafts  resulted  in  no  recommendations. 
Examples  of  editorial  recommendations  included: 

--change  year  from  "1900"  to  "1990," 

--renumber  paragraphs  from  "4.1.1,  4.1.2,  4.1.2,  4.1.3" 
to  "4.1.1,  4.1.2,  4.1.3,  4.1.4," 
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TABLE  8 


Editorial  Recommendations 


DRFP 

Recommendations 

Changes 

PMR 

1 

2 

2 

2 

0 

0 

3 

0 

0 

4 

0 

0 

5 

3 

0 

6 

3 

1 

7 

1 

0 

8 

8 

4 

9 

2 

0 

10 

10 

1 

11 

0 

0 

12 

7 

1 

36 

9 

AE/RW 

1 

33 

11 

2 

0 

0 

3 

9 

4 

4 

0 

0 

5 

27 

18 

69 

33 

105 

42 

--merge  paragraph  4.1.5  with  4.2, 

--incorrect  title  to  paragraph, 

--Table  II  -  "x"  in  wrong  column, 

--change  Roman  numerals  to  Arabic,  and 
--change  verb  usage  to  future  tense. 

Personal  Interviews 

Personal  interviews  were  conducted  in  support  of 
Research  Questions  3.1  and  3.2  to  explore  which  factors  decrease 
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and  which  factors  increase  the  effectiveness  of  the  DRFP  con¬ 


cept.  Additionally,  the  questions  provided  data  to  distin¬ 
guish  procedural  differences  in  support  of  Research  Question 
2.2.  As  discussed  in  Chapter  II,  PCOs  and  buyers  provided 
the  responses  to  questions.  The  data  collected  provide  in¬ 
sight  into  factors  which  affect  the  effectiveness  of  DRFPs 
from  individuals  who  have  actual  experience  with  the  DRFP 
process.  Interview  questions  are  included  in  the  interview 
schedule  (Appendix  C) .  The  questions  and  responses  are  pre¬ 
sented  in  the  order  in  which  they  appear  in  the  schedule. 

Question  #1.  What  is  the  length  of  your  R§D  Con- 
tracting/SPO  experience? 

The  purpose  of  this  question  was  to  establish  the 
level  of  experience  of  the  interviewees.  The  experience 
level  ranged  from  three  to  30  years.  As  indicated  in  Table 
9,  the  majority  of  the  interviewees  have  10  or  more  years  of 
contracting  experience. 

Question  #2.  How  many  DRFPs  have  you  issued? 

This  question  was  designed  to  ascertain  the  inter¬ 
viewees'  experience  with  the  DRFP  process.  Forty- two  percent 
of  the  interviewees  issued  from  one  to  four  DRFPs  and  another 
42  percent  issued  from  five  to  nine  drafts.  Table  10  matches 
the  number  of  individuals  with  the  number  of  DRFPs  issued. 

Question  #3.  Do  you  feel  the  requirement  for  DRFPs 
should  be  based  on  dollar  threshold,  nature  of  the  acquisition, 
or  some  combination  of  the  two?  Why? 

In  their  responses,  100  percent  of  the  interviewees 
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TABLE  9 

Level  of  Experience 

Number 

of  Years 

Number  of  Individuals 

1 

-  4 

2 

5 

-  9 

2 

10 

-  19 

6 

20 

-  30 

2 

TABLE  10 
DRFPs  Issued 


remarked  that  the  requirement  for  the  submittal  of  DRFPs  should 
not  be  based  strictly  on  dollar  threshold.  All  responses 
specified  that  the  nature  of  the  acquisition  should  be  a  major 
consideration  in  determining  the  need  for  a  DRFP.  In  addition, 
they  expressed  the  need  for  consideration  of  the  potential 
benefit  to  be  received  from  industry's  comments.  The  follow* 
ing  is  a  sample  of  the  responses  given: 

--prioritize  and  allow  the  technical  nature  of  the  program 
the  authority  to  dictate  the  requirement  for  a  DRFP 
since  dollars  merely  represent  the  levels  of  difficulties 
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in  technology. 


--DRFPs  should  be  used  for  competitive  type  acquisitions 
versus  sole  source, 

--the  requirement  for  a  DRFP  should  be  determined  by  the 
PCO, 

--the  requirement  for  a  DRFP  should  be  based  on  a  combina¬ 
tion  of  the  nature  of  the  acquisition  and  whether  or 
not  the  project  engineer  feels  that  there  is  benefit 
to  be  received  from  industry's  comments, 

--the  requirement  should  be  based  on  a  combination  of  the 
two  because  some  acquisitions  are  low  in  dollar  thres¬ 
hold;  however,  there  is  a  great  deal  to  be  gained  from 
industry's  comments,  and 

--some  DRFPs  are  issued  only  because  they  meet  the  dollar 
threshold;  in  some  cases  the  engineers  are  firm  about 
their  requirements,  and  there  are  no  clear  benefits  to 
be  gained  from  industry's  comments. 

Question  #4.  Do  you  feel  the  DRFP  process  affects 
the  length  of  the  acquisition  cycle?  What  is  the  impact? 

This  question  delved  into  the  interviewees'  percep¬ 
tions  of  the  DRFP's  impact  upon  the  length  of  the  acquisition 
cycle.  One-half  of  the  responses  indicated  that  the  DRFP 
increases  the  length  of  the  cycle.  Another  one -third  responded 
that  the  DRFP  can  add  to  the  length  depending  upon  the  proce¬ 
dures.  Finally,  two  respondents  perceived  no  impact  upon  the 
acquisition  cycle.  Those  who  felt  that  the  DRFP  increased 
the  length  estimated  that  it  added  from  15  to  60  days.  The 


dominant  factor  in  not  increasing  the  length  was  advance 
planning  and  the  submission  of  the  DRFP  package  to  contract¬ 
ing  personnel  prior  to  forwarding  the  purchase  request. 
However,  numerous  respondents  noted  that  engineers  fail  to  do 
this  in  many  instances.  Another  significant  impression  of 
the  DRFP  recognized  a  balancing-out  of  the  DRFP's  impact  in 
the  long-run.  The  following  are  comments  which  reflect  the 
main  perceptions: 

--the  DRFP  would  not  affect  the  length  if  personnel  have 
accomplished  advanced  planning, 

--the  impact  depends  upon  procedures;  some  engineers 
send  the  DRFP  package  early,  and  others  send  formal 
purchase  requests  without  a  DRFP, 

--the  initial  time  required  for  issuance  results  in  need¬ 
ing  less  time  for  technical  fact-finding,  and 
--it  is  a  fact  of  life  that  if  you  did  not  have  to  issue 
a  DRFP,  you  could  start  the  formal  solicitation  sooner. 

Question  #5.  What  sections  of  the  RFP  package  do  you 
feel  provide  the  most  benefit,  in  terms  of  responses,  if  in¬ 
cluded  in  the  DRFP  package? 

This  question  solicited  details  on  the  amount  of  in¬ 
formation  included  in  DRFP  packages  and  how  the  sections  in¬ 
cluded  affect  the  responses.  One  hundred  percent  of  the 
interviewees  include  the  statement  of  work  in  their  DRFP 
packages.  Ninety-one  percent  include  the  Contract  Data 
Requirements  List. 

These  two  sections  represent  the  major  sections 
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included  in  DRFPs.  The  following  sample  of  responses  repre¬ 
sent  the  variety  of  views  expressed  with  reference  to  the 
inclusion  of  sections  of  the  RFP: 

--it  is  impractical  to  include  the  terms  and  conditions 
and  standard  provisions  because  most  of  the  contractors 
are  aware  of  them  from  previous  experience  with  the 
government.  However,  inclusion  may  be  useful  when 
dealing  with  a  contractor  who  is  new  to  government 
contracting , 

--the  SOW  is  the  most  important  item  to  include, 

--no  comijients  were  received  on  the  evaluation  criteria 
and  standard  clauses  when  they  were  included, 

--the  CDRL  indicating  the  schedule  requirements  should  be 
included,  however,  it  is  not  likely  that  it  will  be 
changed  as  a  result  of  industry  comments, 

--only  the  sections  pertaining  to  the  SOW  and  CDRL  are 
meaningful.  Miscellaneous  information  such  as  type  of 
contract  contemplated  and  estimated  manyears  can  be  in¬ 
cluded  in  the  executive  summary  letter.  In  my  experience, 
all  responses  have  come  from  the  contractors'  engineers 
and  have  not  been  coordinated  through  their  contracts 
office,  and 

--in  order  to  make  the  DRFP  beneficial,  as  much  information 
as  possible  should  be  included. 

Question  #6.  Have  you  discovered  reluctance  on  the 
part  of  the  contractor  to  submit  responses  to  DRFPs  containing 
their  state-of-the-art  technology?  If  so,  why? 
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Contractors'  reluctance  to  submit  responses  could 
possibly  limit  the  effectiveness  of  the  DRFP.  Thus,  this 
question  sought  to  determine  the  degree  to  which  contracting 
personnel  detect  such  a  reluctance.  Eighty- three  percent  of 
the  respondents  acknowledged  that  they  had  detected  a  reluc¬ 
tance  by  contractors  to  include  this  information  in  their 
responses.  Two  individuals  stated  that  they  had  not  actu¬ 
ally  seen  any  reluctance  or  could  not  prove  it.  Of  those 
who  responded  affirmatively  to  experiencing  the  contractors' 
reluctance,  supporting  remarks  included: 

--contractors  will  always  be  reluctant  to  submit  anything 
with  state-of-the-art  technology;  manufacturing  secrets 
are  the  key  to  their  survival, 

--with  a  DRFP,  the  government  is  asking  contractors  to 
put  their  cards  on  the  table  before  the  RFP,  but  con¬ 
tractors  do  not  want  to  expose  their  information  before 
the  formal.  RFP  for  fear  of  compromising  their  position 
and  losing  a  competitive  advantage, 

--responsiveness  appears  to  be  based  upon  the  individual 
contractors,  not  the  nature  of  the  program, 

--there  are  certain  contractors  who  always  respond,  but 
they  tailor  the  responses  to  meet  their  specific  capa¬ 
bilities  in  an  attempt  to  gain  a  competitive  advantage, 
and 

--in  sole  source  situations,  the  contractor  is  more  likely 
to  withhold  state-of-the-art  technology  until  contract 
award  and  afterwards  submit  it  under  value  engineering 
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provisions . 

Question  #7.  What  methods  do  you  use  to  protect 
the  contractors'  recommendations  when  acknowledging  receipt 
of  their  responses? 

This  question  was  designed  to  provide  information  on 
the  methods  used  to  protect  data  provided  by  industry  in  their 
responses  to  DRFPs.  In  addition,  we  sought  to  find  out  if 
industry  responses  are  being  acknowledged,  and  if  so,  how 
detailed  are  the  letters  of  acknowledgment.  Data  showed  that 
33  percent  use  no  method  to  acknowledge  receipt  of  industry 
responses.  Fifty  percent  acknowledge  receipt  by  sending  a 
standard  letter  to  all  respondents  which  simply  thanks  them 
for  responding  to  the  DRFP  and  indicates  the  expected  time 
frame  for  the  release  of  the  formal  RFP.  The  following  is 
a  sample  of  the  methods  used  to  protect  data  submitted  in 
response  to  a  DRFP: 

--if  the  data  is  stamped  proprietary,  it  is  treated  as 
proprietary.  No  technical  information  is  included  in 
the  letter  acknowledging  receipt, 

--the  executive  summary  letter  generally  includes  a  state¬ 
ment  which  assures  that  data  submitted  in  response  to  a 
DRFP  will  be  protected  as  warranted, 

--responses  submitted  are  safeguarded  in  the  same  manner 
as  responses  to  formal  RFPs ,  and 
--recommendations  from  responses  which  are  to  be  imple¬ 
mented  in  the  formal  RFP  are  not  included  in  the 
letters  acknowledging  receipt. 
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Seventeen  percent  of  the  interviewees  included  in¬ 
formation  of  a  general  or  technical  nature  in  their  letters 
of  acknowledgment.  However,  precautions  were  taken  to  guard 
against  the  release  of  proprietary  data.  The  following 
represents  their  comments  on  acknowledgment  of  receipt: 

--general  information  on  the  changes  to  be  made  to  the 
formal  RFP  as  a  result  of  responses  received  is  included 
in  the  letter  of  acknowledgment;  however,  no  specific 
or  proprietary  data  are  included,  and 
--individual  letters  identifying  changes  to  be  made  to 
the  formal  RFP  as  a  result  of  their  responses  are  for¬ 
warded  only  to  the  firm  suggesting  the  change. 

Question  #8.  What  strengths  or  weaknesses  can  you 
identify  in  the  current  DRFP  process? 

The  purpose  of  this  question  was  to  gain  some  insight 
into  the  strengths  and  weaknesses  relative  to  the  DRFP  process 
from  individuals  who  have  actual  experience.  Sixty- five  per¬ 
cent  of  the  responses  were  expressed  in  terms  of  weaknesses 
of  the  current  DRFP  process.  The  most  common  responses  are 
included  below: 

--the  requirement  for  a  DRFP  over  a  certain  dollar  thres¬ 
hold  is  a  weakness;  its  use  is  not  automatically  bene¬ 
ficial  in  every  case  over  a  certain  dollar  amount, 
--issuance  of  a  DRFP  results  in  a  major  undertaking  for 
both  the  buyers  and  clerks  if  you  have  to  develop  a 
model  RFP  and  send  it  through  the  committees  for  review, 
--it  is  a  time-consuming  process  and,  in  most  cases,  no 


feedback  or  responses  are  received, 

--the  requirement  is  too  generally  applied, 

--the  major  weakness  is  the  dollar  threshold  criteria, 
--contractor  and  laboratory  personnel  do  not  understand 
what  we  are  trying  to  accomplish  with  the  DRFP  concept, 
--individuals  in  management  positions  fail  to  recognize 
that  the  DRFP  is  a  draft  document  which  merely  communi¬ 
cates  the  future  requirements  of  the  government  to 
industry, 

--it  is  impractical  to  issue  a  DRFP  in  sole  source  situa¬ 
tions  solely  because  it  meets  the  dollar  threshold 
requirement, 

--contractors  feel  that  once  the  DRFP  is  issued,  the  door 
is  closed  on  discussions  between  government  and  industry 
engineers , 

--if  contractors  have  proprietary  data,  they  are  not  going 
to  reveal  it  in  any  depth  in  theii  responses,  and 
--the  DRFP  process  has  become  too  complex. 

Thirty-five  percent  of  the  responses,  some  of  which 
are  included  below,  expressed  strengths  associated  with  the 
DRFP  process: 

--in  cases  when  the  project  engineer  generally  knows  his 
requirements  but  does  not  have  insight  as  to  what  the 
problem  areas  are,  industry's  comments  can  be  helpful, 
--engineers  with  limited  expertise  in  specific  areas  can 
solicit  information  from  industry  experts, 

--industry's  comments  can  result  in  an  improved  SOW,  cost 
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estimate,  and  overall  formal  RFP  package, 

--contractors  can  identify  unnecessary  requirements, 

especially  concerning  data  submissions  since  this  is  an 
area  where  the  government  has  tendencies  to  overstate 
their  requirements, 

--industry  engineers  may  be  aware  of  technical  innovations 
that  government  engineers  are  not  aware  of,  and 
--the  number  of  responses  received  is  a  good  indicator  of 
the  level  of  interest  to  be  expected  when  the  formal  RFP 
is  issued. 

Question  #9.  What  recommendations  do  you  have  for 
improving  the  current  DRFP  process? 

When  asked  for  recommendations,  eleven  of  the  twelve 
responded  with  suggestions.  Two  major  areas  were  the  subject 
of  the  recommendations- -applicability  of  the  DRFP  and  formality 
of  the  procedures.  The  central  theme  in  determining  the  re¬ 
quirements  for  issuing  a  DRFP  was  delegating  authority  to  the 
engineer  and  basing  the  decision  upon  the  nature  of  the  pro¬ 
gram.  Many  respondents  recalled  several  programs  which  in¬ 
volved  technical  questions  and  unfamiliar  requirements  where 
the  engineers  expected  and  received  valuable  feedback.  How¬ 
ever,  they  also  identified  numerous  programs  where  the  require¬ 
ments  and  specifications  were  well  defined,  but  DRFPs  were 
issued  anyway  because  of  the  dollar  threshold. 

Major  suggestions  concerning  the  formality  called 
for  eliminating  administrative  reviews  and  limiting  responsi¬ 
bility  for  issuance  at  the  level  of  the  contracting  officer. 
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Individuals  emphasized  that  the  DRFP  is  an  informal  method  of 
communicating  with  contractors  and  that  it  is  not  contractually 
binding.  They  stated  that  the  review  process  has  evolved  to 
the  point  where  the  DRFP  is  treated  like  a  formal  solicitation. 

Further  recommendations  included: 

--emphasize  importance  of  advance  planning  and  submission 
of  necessary  data  by  engineers, 

--educate  personnel  so  that  everyone  understands  the  pur¬ 
pose  and  procedures  of  the  DRFP,  and 
--facilitate  more  communication  between  government  and 
industry  during  the  DRFP  process. 

Question  #10.  Do  you  have  any  additional  comments  on 
the  DRFP  process? 

This  question  attempted  to  capture  any  additional 
information  which  the  preceding  questions  may  have  failed  to 
solicit.  One-third  of  the  closing  remarks  reflected  negative 
perceptions  of  the  existing  DRFP  procedures.  These  responses 
indicated  that  the  DRFP  concept  was  not  a  very  valuable  con¬ 
tracting  tool.  Reasons  for  this  attitude  were  that  the  DRFP 
is  used  in  many  acquisitions  where  the  required  effort  and 
resources  exceed  the  resulting  benefits.  Some  respondents 
attributed  the  negative  perception  to  the  types  of  programs 
for  which  they  had  issued  the  DRFPs.  Other  reasons  included 
the  excessive  review  procedures  and  makeshift  rules  in  the 
contracting  offices. 

Some  expressed  neither  negative  nor  positive  comments 
but  emphasized  suggestions  and  responses  from  previous 
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questions.  Also,  another  one-third  did  not  have  any  additional 


comments . 

Data  Analysis 

Appendix  D  contains  a  listing  of  the  data  file  for 
computer  analysis,  the  computer  program,  and  the  computer  out¬ 
put.  The  data  file  contained  four  columns  with  the  following 
variables : 

1)  source  of  DRFP  (a  value  of  one  indicated  PMR 
and  a  value  of  two  indicated  AE  and  RW) , 

2)  the  number  of  sections  in  the  DRFP  package, 

3)  the  number  of  changes  which  met  the  operational 
definition  of  effectiveness,  and 

4)  the  estimated  dollar  amount  of  the  contract. 

As  discussed  in  Chapter  II,  regression  analysis  enabled 
the  research  of  the  relationship  between  the  number  of  changes 
and  the  number  of  sections  in  the  DRFP  package  in  support  of 
Research  Question  2.1.  The  SPSS  subprogram,  REGRESSION, 
supplied  the  computer  capability  to  perform  this  test.  The 
required  variables  were  the  independent  variable,  number  of 
sections,  and  the  dependent  variable,  number  of  changes  imple¬ 
mented.  A  computer  output  lists  the  values  of  the  constant 
and  the  variable  for  the  regression  model: 

Y  =■  2.82  +  (.568  X  10'13)X 

With  such  a  negligible  value  for  bj^,  one  can  surmise 
an  insignificant  relationship  between  the  number  of  sections 
and  number  of  changes.  This  was  confirmed  by  the  value  of 
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the  coefficient  of  simple  correlation  which,  annotated  as 
Multiple  R  on  the  computer  output,  was  zero.  This  value 
indicated  the  lack  of  a  relationship;  however,  the  results  of 
the  additional  tests  explained  in  Chapter  II  are  also  pre¬ 
sented. 

The  first  test  to  determine  whether  g^  ■  0  involved 
the  confidence  interval  with  lower  and  upper  bounds  of: 

L  =  b1  -  t(l-a/2;n-2)s(b1) 

U  »  b1  ♦  t(l-a/2;n-2)s(b1) 

The  critical  value  of  t,  extracted  from  the  statistical  table 
in  Appendix  E,  was  1.753  for  an  a/2  of  .05  and  15  degrees  of 
freedom.  Using  the  values  of  b^  and  s(b^)  from  the  computer 
output,  the  resultant  confidence  interval  was  -1.640808  <_  g^ 
1.640808.  Thus  one  can  state  with  90  percent  confidence 
that  g^  lies  between  +  1.640808,  and  Hq  cannot  be  rejected. 

The  second  test  involved  the  standardized  test  sta- 

.17 

tistic.  The  computed  value  of  t*  was  6.07  X  10  ;  since 

|t*|  is  less  than  1.753,  one  cannot  reject  HQ. 

The  final  test  of  g^  involved  a  comparison  of  b^  with 
the  action  limits  which  were  computed  as: 

Ax  «  -1.640808 
A2  -  1.640808 

The  computed  value  of  b^,  .568  X  10 falls  within  the  action 
limits.  Therefore  the  correct  conclusidn  is  Hfl. 

After  analysis  of  the  relationship  between  the  num¬ 
ber  of  changes  and  the  number  of  sections,  the  SPSS  program 
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was  again  executed  with  the  contract  dollar  amount  as  the 
independent  variable  to  explore  the  relationship  between  the 
number  of  changes  and  contract  dollar  amount.  This  research 
exceeded  the  intended  scope  of  the  methodology;  however,  it 
followed  the  same  design  as  the  evaluation  of  the  relation¬ 
ship  between  number  of  changes  and  sections.  As  given  in 
Appendix  D,  the  values  of  the  constant  and  variable  for  the 
regression  model  were: 

Y  *  2.61  +  . 000018X 

Once  more  there  was  an  insignificant  value  for  b^,  but  the 
value  for  the  coefficient  of  simple  correlation  was  .188, 
which  signified  an  existing  but  weak  relationship. 

The  test  of  whether  g^  *  0  began  with  the  confidence 
interval  discussed  in  Chapter  II.  The  lower  and  upper  bounds 
were : 

L  *  -.00002 
U  *  .00006 

Therefore,  the  resulting  interval  is  -.00002  <_  g^  <  .00006, 
and  Hq  cannot  be  rejected. 

The  computed  value  of  the  standardized  test  statis¬ 
tic  was  .743,  which  was  less  than  1.753;  thus  one  must  conclude 
Hq. 

The  action  limits,  calculated  according  to  the  for¬ 
mulae  of  Chapter  II,  were: 

Aj_  ■  0  -  (1. 753)  ( .  2468424X10  ”4)  -  -.00004 
A2  -  0  +  (1.753)(.2468424X104)  «  .00004 


56 


f 


As  shown  in  Appendix  D,  the  value  of  was  .000018;  thus  the 
results  support  Hq. 

In  analyzing  the  data  for  Research  Question  2.2,  the 
T-Test  provided  the  statistical  method  of  evaluating  whether 
there  exists  a  difference  in  the  results  of  the  DRFP  process 
between  PMR  vis-a-vis  AE  and  RW.  As  presented  in  Chapter  II, 
the  hypotheses  were: 

H0:  U2  "  yl  *  0  (no  difference  exists) 

H^:  U2  -  f  0  (difference  exists) 

The  first  test  involved  a  confidence  interval  based 
upon  an  a/2  of  .05  and  15  degrees  of  freedom.  The  critical 
value  of  t,  extracted  from  the  statistical  table  in  Appendix 
E,  was  1.753  and  the  resultant  confidence  interval  was 
-6.0572  y 2  '  i  .4572.  Since  this  captured  the  value  of 

zero,  the  null  hypothesis  could  not  be  rejected. 

The  probability  value  was  extracted  from  the  pooled 
variance  estimate  since  equal  variances  were  initially  assumed. 
This  value  of  .153  was  greater  than  the  a  value  of  .10;  there¬ 
fore,  the  appropriate  conclusion  is  Hq. 

The  action  limits  for  comparison  with  y2  •  were: 

Al  -  -3.26 
A2  -  3.26 

Since  D  ■  -2.8,  the  proper  conclusion  was  Hq. 

The  last  test  involved  a  comparison  of  the  t  value 
from  the  computer  output  and  the  critical  value  of  t.  Again, 

Hq  was  the  correct  conclusion  since  1.51,  the  computed  value, 
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is  less  than  1.753. 

The  next  chapter  presents  the  findings  and  recom¬ 
mendations  which  have  resulted  from  the  research  effort. 


CHAPTER  IV 


FINDINGS  AND  RECOMMENDATIONS 

This  chapter  presents  the  findings  of  the  research 
questions  listed  in  Chapter  I.  Along  with  the  findings,  con¬ 
clusions  are  given  concerning  the  efforts  of  the  research. 
Additionally,  recommendations  for  application  of  the  research 
findings  and  suggestions  for  further  research  are  given. 

Findings 

The  research  findings  are  presented  in  the  order  of 
the  research  questions.  As  discussed  in  earlier  chapters, 
specific  objectives  included: 

--describing  the  objectives  and  procedures  of  the  DRFP 
concept , 

--describing  the  effectiveness  of  the  DRFP  concept, 
--evaluating  differences  in  the  procedures  and  results  of 
the  DRFP  in  different  phases  of  the  acquisition  process,  and 
--determining  factors  which  decrease  or  increase  the 
effectiveness  of  the  DRFP  concept. 

Research  Questions  1.1/1. 2 

What  are  the  objectives  of  the  DRFP  concept?  What 
are  the  procedures  for  implementing  the  DRFP  concept? 

As  discussed  in  Chapter  III,  the  review  of  defense 
documents  provided  the  objectives  and  procedures  for 
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implementing  the  DRFP  concept.  The  major  sources  of  guidance 
are  AFSC  DAR  Supplement  3-550  and  AFSC  Pamphlet  70-4,  which 
specify  procedures  that  are  generally  applicable  to  AFSC.  As 
presented  earlier,  the  purpose  of  the  DRFP  concept  is  to  pro¬ 
vide  significant  cost  savings  and  improvements  in  planned 


acquisitions  through  industry  feedback.  The  general  procedures 
utilized  to  accomplish  the  objectives  of  the  DRFP  concept  call 
for  advance  planning  and  early  interaction.  The  research 
recognized  this  requirement  to  be  the  key  to  effectiveness. 
Overall,  the  research  revealed  that  the  stated  procedures  for 
implementing  the  DRFP  concept  are  general  in  nature  and  exist 
at  AFSC  level. 

Research  Question  2.1 

How  effective  is  the  DRFP  concept  in  achieving  its 
stated  objective? 

The  contract  data,  presented  in  Chapter  III,  were  col¬ 
lected  by  using  the  operational  definition  of  effectiveness  to 
identify  contractor  recommendations  which  achieved  the  stated 
objective  of  the  DRFP  concept.  As  mentioned  earlier,  35  per¬ 
cent  of  the  contractors'  recommendations  in  PMR  DRFPs  resulted 
in  changes  which  satisfied  the  operational  definition,  and  18 
percent  of  the  recommendations  met  the  definition  in  AE/RW 
DRFPs.  The  research  also  annotated  two  categories  of  responses 
which  did  not  fall  within  the  operational  definition.  The 
first  category  grouped  responses  which  addressed  general  prob¬ 
lem  areas  in  an  attempt  to  capture  elements  of  effectiveness 


which  are  not  formally  recognized.  A  valuable  function  of  the 
DRFP  was  identified  with  more  than  50  percent  of  the  changes 
addressing  the  need  to  clarify  or  define  terms  and  require¬ 
ments.  The  second  category  identified  responses  which  serve 
an  editorial  function  in  the  DRFP;  this  area  revealed  some 
contractor  attention  to  areas  which  do  not  contribute  to 
effectiveness.  Although  two  additional  elements  were  added 
to  the  operational  definition  of  effectiveness  for  data  col¬ 
lection,  the  researchers  did  not  find  that  these  added  to 
effectiveness . 

In  analyzing  the  effectiveness  of  the  DRFP,  the  re¬ 
search  evaluated  the  relationship  between  the  effectiveness 
of  the  DRFP  and  the  number  of  sections  in  the  DRFP  package. 

As  inferred  by  the  results  of  the  statistical  .tests  in  Chapter 
III,  no  relationship  was  discovered.  This  conclusion  was 
substantiated  by  responses  to  question  five  of  the  personal 
interview  which  indicated  that  the  most  productive  sections 
are  the  description/specifications  and  CDRL. 

In  summary,  the  research  did  not  attempt  to  yield 
a  final  evaluation  of  whether  the  DRFP  is  effective  or  not. 
However,  the  findings  resulted  in  a  description  of  the  effec¬ 
tiveness  which  result  primarily  from  inclusion  of  the 
description/specifications  and  CDRL.  Additionally,  the 
effectiveness  has  been  enhanced  by  recommendations  which  iden¬ 
tify  terms  and  requirements  needing  clarification  or  defini¬ 
tion. 


Research  Question  2.2 

Is  there  a  difference  in  the  procedures  for  using 
the  DRFP  and  the  results  of  the  DRFP  in  the  different  phases 
of  the  acquisition  process? 

The  research  did  not  result  in  a  distinction  between 
procedures  in  PMR  vis-a-vis  AE  and  RW.  However,  responses  to 
questions  in  the  personal  interview  and  efforts  to  identify 
contracts  for  analysis  revealed  that  there  are  differences  in 
procedures  among  contracting  officers  in  each  organization. 

One  of  the  primary  differences  was  the  lack  or  presence  of 
advance  planning  and  the  submission  of  the  DRFP  package  by 
engineers.  This  was  a  significant  factor  in  the  DRFP's  impact 
on  the  length  of  the  acquisition  cycle.  Another  area  with 
prominent  differences  was  the  acknowledgment  of  contractor 
recommendations.  As  discussed  in  Chapter  III,  one- third  made 
no  form  of  acknowledgment,  50  percent  addressed  standard 
letters  to  all  respondents  which  acknowledged  receipt  and 
provided  the  general  time  frame  for  issuance  of  the  formal  RFP, 
and  17  percent  included  information  of  a  general  or  technical 
nature. 

The  analysis  of  the  research  data  also  revealed  that 
there  were  not  any  significant  differences  in  the  results  of 
the  DRFP  concept  between  PMR  and  AE/RW.  This  conclusion  was 
derived  from  the  test  results  shown  in  Chapter  III  at  a  90 
percent  confidence  level.  Thus,  the  DRFP  concept  results  in 
comparable  levels  of  effectiveness  among  the  different  phases 
of  the  acquisition  process. 
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Research  Questions  3. 1/3. 2 

What  factors  decrease  the  effectiveness  of  the  DRFP 
concept?  What  factors  increase  the  effectiveness  of  the  DRFP 
concept? 

The  purpose  of  the  DRFP  concept  is  to  solicit  industry 
feedback  on  those  areas  of  planned  acquisitions  that  may  pro¬ 
vide  significant  cost  savings  and  program  improvement;  however, 
there  are  factors  which  affect  the  achievement  of  this  purpose. 

A  significant  majority  of  contracting  officers  detect 
reluctance  by  contractors  to  include  state-of-the-art  informa¬ 
tion  in  their  responses.  In  addition,  the  majority  of  contrac¬ 
ting  personnel  perceive  important  weaknesses  in  the  current 
DRFP  process.  The  most  commonly  stated  weaknesses  related  to 
the  additional  time  required  to  accomplish  a  DRFP,  the  failure 
to  clearly  understand  the  objectives  of  the  DRFP  concept,  and 
the  general  applicability  of  the  DRFP  due  to  the  dollar  thres¬ 
hold  criteria. 

However,  there  are  also  strengths  associated  with 
the  DRFP  process  which  include  1)  the  submission  of  expert 
information  from  industry  personnel  in  areas  where  government 
personnel  have  limited  knowledge,  2)  the  identification  of 
unnecessary  requirements,  3)  an  indication  of  the  potential 
level  of  interest,  and  4)  identification  and  coordination  of 
technical  innovations. 

The  negative  perceptions  which  have  been  attributed 
to  the  excessive  review  procedures,  lack  of  standardization 
of  procedures,  and  general  applicability  of  the  DRFP  all 
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contribute  to  a  decrease  in  the  effectiveness  of  the  DRFP 
concept.  On  the  other  hand,  strengths  expressed  by  the  inter¬ 
viewees  are  factors  that  contribute  to  increasing  the  effec¬ 
tiveness  of  the  DRFP  concept  and  provide  support  for  the 
overall  achievement  of  its  objectives. 

Corollary  Findings 

As  discussed  under  Research  Question  2.1,  the  research 
revealed  that  no  relationship  existed  between  the  effectiveness 
of  the  DRFP  and  the  completeness  of  the  DRFP  package.  Anti¬ 
cipating  that  some  may  hypothesize  a  relationship  between  the 
effectiveness  and  proposed  dollar  amount,  we  also  tested  this 
relationship.  Again,  the  research  revealed  the  absence  of  a 
relationship.  However,  the  responses  to  the  personal  inter¬ 
views  indicated  that  the  success  or  effectiveness  of  the  DRFP 
in  achieving  its  objectives  is  based  upon  the  nature  of  the 
acquisition.  The  use  of  DRFPs  in  programs  which  involve  a 
high  degree  of  complexity  or  lack  sufficient  definitude  result 
in  higher  levels  of  effectiveness. 

Another  area  of  concern  identified  by  the  research 
is  an  insufficient. understanding  of  the  objectives  and  proce¬ 
dures  of  the  DRFP  concept.  The  major  sources  of  misunderstand¬ 
ing  are  evidenced  first  by  failure  of  engineers  to  submit 
necessary  information  early  in  the  process.  Additionally, 
many  of  the  responses  by  industry  fail  to  fulfill  the  objec¬ 
tives  of  the  DRFP  concept;  rather,  it  appears  that  in  some 
cases  contractors  use  the  DRFP  to  advertise  their  unique  capa¬ 
bilities. 
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The  third  finding  involved  the  formality  of  the  DRFP 
process  which  has  evolved  since  its  inception.  The  present 
process  includes  numerous  reviews  which  treat  the  DRFP  as  a 
formal  document  rather  than  as  an  informal  method  of  communi¬ 
cating  with  industry. 

Recommendations  for  Application 

The  following  are  recommendations  derived  from  the 
research  effort: 

1)  ASD  should  establish  guidelines  which  simplify 
and  provide  flexibility  for  accomplishing  the  objectives  of 
the  DRFP  concept.  The  level  of  complexity  involved  in  prepar¬ 
ing  DRFPs  >  numerous  reviews ,  and  formal  treatment  of  the  DRFP 
are  factors  which  decrease  the  overall  effectiveness  of  the 
DRFP  concept. 

2)  Criteria  for  the  requirement  of  DRFPs  should  be 
re-evaluated.  The  nature  of  the  acquisition  with  reference 
to  level  of  complexity  and  the  definition  of  requirements  may 
be  a  better  indicator  of  programs  which  are  prime  candidates 
for  DRFPs. 

3)  Contracting  officers  should  not  include  specific 
contractor  recommendations  in  letters  acknowledging  receipt 
of  responses.  This  change  will  prevent  early  disclosure  of 
the  contractor's  state-of-the-art  technology. 

4)  The  objectives  of  the  DRFP  should  include  the 

identification  of  terms  and  requirements  which  require  greater 
clarification. 


5)  A  current  orientation  of  the  objectives  and  proce¬ 
dures  of  the  DRFP  concept  should  be  provided  to  engineers  and 
defense  contractors. 

6)  Engineers  should  play  a  greater  role  in  determin¬ 
ing  the  need  for  a  DRFP.  They  are  aware  of  the  level  of  ex¬ 
pertise  available,  the  in-house  capabilities,  and  the  flexi¬ 
bility  of  requirements.  This  knowledge  places  them  in  a  better 
position  to  determine  the  need  for  industry's  input. 

Recommendations  for 
Further  Study 

This  research  was  accomplished  to  provide  an  initial 
study  of  the  DRFP  concept  and  its  results  have  provided  a  des¬ 
criptive  level  of  analysis.  In  order  to  validate  the  findings 
and  generalize  beyond  ASD,  it  is  recommended  that  the  same 
methodology  be  used  for  research  in  other  organizations  of 
AFSC.  This  research  would  provide  a  description  of  the  DRFP's 
scope  of  application  in  AFSC  and  describe  factors  affecting 
its  effectiveness. 

Another  area  of  future  research  involves  a  comparative 
analysis  of  contracts  using  DRFPs  from  a  data  source,  identi¬ 
fied  by  engineers,  which  comprise  programs  of  a  well-defined 
nature  versus  complex  and  undefined  programs.  Such  research 
would  evaluate  the  effectiveness  of  the  DRFP  concept  in  terms 
of  the  nature  of  the  program. 

Finally,  research  using  questionnaires  and  interviews 


of  government  engineers  and  industry  representatives  would 
provide,  from  another  perspective,  factors  which  decrease  or 
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increase  the  effectiveness  of  the  DRFP  and  also  provide  recoin 
mendations  for  improvement. 
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APPENDIX  A 

RESEARCH  GUIDE  AND  WORKSHEET  FOR 
DRFP  DATA  EXTRACTION 
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RESEARCH  GUIDE 


I.  Operational  Definition  of  Effectiveness 

An  effective  DRFP  is  one  that  results  in  implementa¬ 
tions  which  address  the  areas  of  potential  cost  savings. 

These  areas  include: 

1)  Changes  to  requirements  specifying  manufacturing  pro¬ 
cesses,  facilities,  tools,  and  test  capabilities, 

2)  Schedule  or  delivery  changes  that  will  result  in  cost 
reductions , 

3)  Substitution  of  RFP-stated  materials;  and  overly  re¬ 
strictive  engineering  and  quality  specifications  and 
requirements , 

4)  The  use  of  common  government -furnished  equipment  or 
standard  commercial  components  instead  of  system 
peculiar  items, 

5)  Reduction,  substitution,  or  deletion  of  military 
specifications  and  standards, 

6)  Excessive  reporting  requirements  (to  include  data 
requirements) , 

7)  Modified  contractual  arrangements  designed  to  finan¬ 
cially  incentivize  contractors  and  subcontractors  to 
pursue  cost-reduction  efforts  during  contract  per¬ 
formance  , 

8)  Identification,  simplification,  or  reduction  of  special 
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clauses , 

9)  Compatibility  of  contract  type  with  statement  of  work. 

II.  For  each  contract,  document.  .  . 

A.  DRFP  number/buyer 

B.  Estimated  dollar  amount 

C.  Type  of  contract 

D.  Number  of  responses  solicited 

E.  Number  of  responses  received 

F.  Sections  included  in  DRFP 

G.  Contractor  responses  which  address  general  problem 
areas  other  than  specific  areas  of  operational  defi¬ 
nition  of  effectiveness 

H.  Contractor  recommendations  which  address  one  or  more 
of  the  nine  areas  in  the  operational  definition  of 
effectiveness  (Document  recommendation  and  applicable 
section) 

I.  Additional  contractor  recommendations  which  are  of 
an  editorial  nature  (Document  recommendation  and 
applicable  section) 


APPENDIX  B 
FORMULAE  FOR  T-TEST 
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1.  Determine  the  mean  of  the  sampling  distribution  where: 


E(D)  =  U2  -  Hi 

2,  Determine  the  estimate  of  the  common  population  variance 
where: 

2  _  *  (n2-l)s| 

sc  '  (nj-r)  V  (n2-l) 

2 

3.  Calculate  the  unbiased  estimator  of  a  (D)  where: 


•2<®>  *  sc  +  ^ 

*  the  mean  of  changes  in  AE/RW  DRFPs 
^2  ~  the  mean  of  changes  in  PMR  DRFPs 
n^  *  the  sample  size  of  AE/RW  DRFPs 
n2  *  the  sample  size  of  PMR  DRFPs  (16:313) 
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APPENDIX  C 
INTERVIEW  SCHEDULE 
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This  is  the  final  phase  of  research  for  an  Air  Force 
Institute  of  Technology  thesis  on  the  effectiveness  of  the 
Draft  Request  for  Proposal  (DRFP) .  We  have  reviewed  solici¬ 
tation  packages  of  contracts  which  included  a  DRFP  to  deter¬ 
mine  the  scope  of  the  DRFP  package  and  the  number  of  industry 
responses  implemented  in  the  formal  RFP.  However,  much  of 
our  effort  requires  insight  into  the  knowledge  and  experience 
of  persons,  like  yourself,  involved  in  the  acquisition  process. 
Therefore,  we  appreciate  the  time  you  are  allowing  us  for 
this  interview. 


Date 


Individual  interviewed 
Current  title/office 


1.  What  is  the  length  of  your  R$D  Contracting/SPO  experience? 


2.  How  many  DRFPs  have  you  issued? 


3.  Do  you  feel  the  requirement  for  DRFPs  should  be  based  on 
dollar  threshold,  nature  of  the  acquisition,  or  some  com 
bination  of  the  two?  Why? 


4.  Do  you  feel  the  DRFP  process  affects  the  length  of  the 
acquisition  cycle?  What  is  the  impact? 


5.  What  sections  of  the  RFP  package  do  you  feel  provide  the  most 
benefit,  in  terms  of  responses,  if  included  in  the  DRFP 
package? 
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6.  Have  you  discovered  reluctance  on  the  part  of  the  contrac¬ 
tor  to  submit  responses  to  DRFPs  containing  their  state- 
of-the-art  technology?  If  so,  why? 


7.  What  methods  do  you  use  to  protect  the  contractors'  recom¬ 
mendations  when  acknowledging  receipt  of  their  responses? 


8.  What  strengths  or  weaknesses  can  you  identify  in  the  cur¬ 
rent  DRFP  process? 


9.  What  recommendations  do  you  have  for  improving  the  current 
DRFP  process? 


10.  Do  you  have  any  additional  comments  on  the  DRFP  process? 
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OEPENOENT  VARIABLE.. 


I*P 

MEAN  RESPONSE  2.32TB3  STO.  OEV.  3.32633 

VARIABLES)  ENTERED  ON  5T*P  N'J*19ER  1..  SEC 


MULTIPLE  R  * 
R  SQUARE  3 
AO JUST EO  R  SQUARE  9 
STO  DEVIATION  3.745)5 


F  SIGNIFICANCE 

C  1.000 


ANALYSIS  OF  VARIANCE 

REGRESSION 

RESIDUAL 

COEFF  OF  VARIABILITY 


OF  SUM  OF  SQUARES  MEAN  SQUARE 

1*  0  C 

15.  210.47059  14.03137 

132.7  PIT 


-  VARIABLES  IN  THE  EQUATION  - 


VARIABLE  B  STO  ERROR  B  F 

SIGNIFICANCE 

SEC  .5S843419E-13  .93646ie4  .36845143 £-25 

li‘JV) 

(CONSTANT)  2.3235294  2.9526297  .91446413 

.  354 


ALL  VARIABLES  ARE  IN  HE  EQUATION. 
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[DEPENDENT  VARIABLE..  Hp 

MEAN  RESPONSE  2.32353  STD.  DEV.  3,62693 

VARIABLE(S)  ENTEREO  ON  3TED  N'JM^EA  OOL 


MULTIPLE  R  .133*2 
R  SQUARE  .33553 
AO JUST EO  R  SOUARE  0 
STD  DEVIATION  3.573'& 


F  SIGNIFICANCE 

•55212  .469 


ANALYSIS  OF  VARIANCE 

OF 

SUM  OF  SQUARES 

mean  square 

REGRESSION 

1. 

7.47199 

7.47199 

RESIDUAL 

15  • 

23  2. 99850 

13.5  3324 

COEFF  OF  VARIABILITY 

130.3 

3CT 

VARIABLE 

OOL 

(CONSTANT) 


........  VARt  A3LES  IN  THE  EQUATION - 

g  STO  ERROR  B  F 

SIGNIFICANCE 

•  1 33<*i594£»a*  .  2468424')E-04  .55212147 

•  469 

2*  6115627  .93674745  7.7720562 

*314 


ALL  VARIABLES  ARE  IN  THE  EQUATION. 
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